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Preface 



The current control problems present natural trend of increasing its complexity due to 
performance criteria that is becoming more sophisticated. The necessity of practicers 
and engineers in dealing with complex dynamic systems has motivated the design of 
controllers, whose structures are based on multiobjective constraints, knowledge from 
expert, uncertainties, nonlinearities, parameters that vary with time, time delay 
conditions, multivariable systems, and others. The classic and modern control theories, 
characterized by input-output representation and state-space representation, 
respectively, have contributed for proposal of several control methodologies, taking 
into account the complexity of the dynamic system. Nowadays, the explosion of new 
technologies made the use of computational intelligence in the controller structure 
possible, considering the impacts of Neural Networks, Genetic Algorithms, Fuzzy 
systems, and others tools inspired in the human intelligence or evolutive behavior. 
The fusion of classical and modern control theories and the computational intelligence 
has also promoted new discoveries and important insights for proposal of new 
advanced control techniques in the context of robust control, adaptive control, optimal 
control, predictive control and intelligent control. These techniques have contributed 
to a successful implementations of controllers and obtained great attention from 
industry and academy to propose new theories and applications on advanced control 
systems. 

In recent years, the control theory has received significant attention from the academy 
and industry so that researchers still carry on making contribution to this emerging 
area. In this regard, there is a need to publish a book covering this technology. 
Although there have been many journal and conference articles in the literature, they 
often look fragmental and messy, and thus are not easy to follow up. In particular, a 
rookie who plans to do research in this field can not immediately keep pace to the 
evolution of these related research issues. This book, Frontiers in Advanced Control 
Systems, pretends to bring the state-of-art research results on advanced control from 
both the theoretical and practical perspectives. The fundamental and advanced 
research results as well as the contributions in terms of the technical evolution of 
control theory are of particular interest. 

Chapter one highlights some aspects on fuzzy model based advanced control systems. 
The interest in this brief discussion is motivated due to applicability of fuzzy systems 
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to represent dynamic systems with complex characteristics such as nonlinearity, 
uncertainty, time delay, etc., so that controllers, designed based on such models, can 
ensure stability and robustness of the control system. Finally, experimental results of a 
case study on adaptive fuzzy model based control of a multivariable nonlinear pH 
process, commonly found in industrial environment, are presented. 

Chapter two brings together cooperative control, reinforcement learning, and game 
theory to solve multi-player differential games on communication graph topologies. 
The coupled Riccati equations are developed and stability and solution for Nash 
equilibrium are proven. A policy iteration algorithm for the solution of graphical 
games is proposed and its convergence is proven. A simulation example illustrates the 
effectiveness of the proposed algorithms in learning in real-time, and the solutions of 
graphical games. 

Chapter three presents an application of adaptive neural networks to the estimation of 
the product compositions in a binary methanol-water continuous distillation column 
from available temperature measurements. A software sensor is applied to train a 
neural network model so that a GA performs the search for the optimal dual control 
law applied to the distillation column. Experimental results of the proposed 
methodology show the performance of the designed neural network based control 
system for both set point tracking and disturbance rejection cases. 

Chapter four proposes new methods for optimizing the controller's norm, considering 
different criteria of stability, as well as the inclusion of a decay rate in LMIs 
formulation. The 3-DOF helicopter practical application shows the advantage of the 
proposed method regarding implementation cost and required effort on the motors. 
These characteristics of optimality and robustness make the design methodology 
attractive from the standpoint of practical applications for systems subject to structural 
failure, guaranteeing robust stability and small oscillations in the occurrence of faults. 

Chapter five presents a study about the stability and control design for switched affine 
systems. A new theorem for designing switching affine control systems, is proposed. 
Finally, simulation results involving four types of converters namely Buck, Boost, 
Buck-Boost and Sepic illustrate the simplicity, quality and usefulness of the proposed 
methodology. 

Chapter six proposes a new method of model based PID controller tuning for a large 
class of processes (stable processes, processes having oscillatory dynamics, integrating 
and unstable processes), in a classification plane, to guarantee the desired 
performance/robustness tradeoff according to parameter plane. Experimental results 
show the advantage and efficiency of the proposed methodology for the PID control of 
a real thermal plant by using a look-up table of parameters. 

In chapter seven, Bio-inspired Optimization Methods (BiOM) are used for controllers 
tuning in chemical engineering problems. For this finality, three problems are studied, 



Preface XI 

with emphasis on a realistic application: the control design of heat exchangers on pilot 
scale. Experimental results show a comparative analysis with classical methods, in the 
sense of illustrating that the proposed methodology represents an interesting 
alternative for this purpose. 

In chapter eight, a novel method for centralized-decentralized coordinated cooperative 
control of multiple wheeled mobile manipulators, is proposed. In this strategy, the 
desired motions are specified as a function of cluster attributes, such as position, 
orientation, and geometry. These attributes guide the selection of a set of independent 
system state variables suitable for specification, control, and monitoring. The control is 
based on a virtual 3-dimensional structure, where the position control (or tracking 
control) is carried out considering the centroid of the upper side of a geometric 
structure (shaped as a prism) corresponding to a three-mobile manipulators formation. 
Simulation results show the good performance of proposed multi-layer control 
scheme. 

Chapter nine proposes a Model Predictive Control (MPC) strategy, formulated under a 
stabilizing control law assuming that this law (underlying input sequence) is present 
throughout the predictions. The MPC proposed is an Infinite Horizon MPC (IHMPC) 
that includes an underlying control sequence as a (deficient) reference candidate to be 
improved for the tracking control. Then, by solving on line a constrained optimization 
problem, the input sequence is corrected, and so the learning updating is performed. 

Chapter ten has its focus on the PID average output feedback controller, implemented 
in an FPGA, to stabilize the output voltage of a "buck" power converter around a 
desired constant output reference voltage. Experimental results show the effectiveness 
of the FPGA realization of the PID controller in the design of switched mode power 
supplies with efficiency greater than 95%. 

Chapter eleven aims at discussing parameter estimation techniques to generate 
suitable models for predictive controllers. Such discussion is based on the most 
noticeable approaches in Model Predictive Control (MPC) relevant identification 
literature. The first contribution to be emphasized is that these methods are described 
in a multivariable context. Furthermore, the comparisons performed between the 
presented techniques are pointed as another main contribution, since it provides 
insights into numerical issues and exactness of each parameter estimation approach 
for predictive control of multivariable plants. 

Chapter twelve presents a contribution for systems identification using Orthonormal 
Basis Filter (OBF). Considerations are made based on several characteristics that make 
them very promising for system identification and their application in predictive 
control scenario. 

This book can serve as a bridge between people who are working on the theoretical 
and practical research on control theory, and facilitate the proposal for development of 
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new control techniques and its applications. In addition, this book presents 
educational importance to help students and researchers to know the frontiers in 
control technology. The target audience of this book can be composed of professionals 
and researchers working in the fields of automation, control and instrumentation. 
Book can provide to the target audience the state-of-art in control theory from both the 
theoretical and practical aspects. Moreover, it can serve as a research handbook on the 
trends in the control theory and solutions for research problems which requires 
immediate results. 



Prof. Ginalber Luiz de Oliveira Serra 

Federal Institute of Education, Sciences and Technology, 

Brazil 
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Highlighted Aspects from Black Box Fuzzy 
Modeling for Advanced Control Systems Design 

Ginalber Luiz de Oliveira Serra 

Federal Institute of Education, Science and Technology 
Laboratory of Computational Intelligence Applied to Technology, Sao Luis, Maranhao 

Brazil 

1. Introduction 

This chapter presents an overview of a specific application of computational intelligence 
techniques, specifically, fuzzy systems: fuzzy model based advanced control systems design. 
In the last two decades, fuzzy systems have been useful for identification and control of 
complex nonlinear dynamical systems. This rapid growth, and the interest in this discussion 
is motivated by the fact that the practical control design, due to the presence of nonlinearity 
and uncertainty in the dynamical system, fuzzy models are capable of representing the 
dynamic behavior well enough so that the real controllers designed based on such models 
can garantee, mathematically, stability and robustness of the control system (Astrom et al., 
2001; Castillo-Toledo & Meda-Campana, 2004; Kadmiry & Driankov, 2004; Ren & Chen, 2004; 
Tong & Li, 2002; Wang & Luoh, 2004; Yoneyama, 2004). 

Automatic control systems have become an essential part of our daily life. They are applied 
in an electroelectronic equipment and up to even at most complex problem as aircraft and 
rockets. There are different control systems schemes, but in common, all of them have 
the function to handle a dynamic system to meet certain performance specifications. An 
intermediate and important control systems design step, is to obtain some knowledge of the 
plant to be controlled, this is, the dynamic behavior of the plant under different operating 
conditions. If such knowledge is not available, it becomes difficult to create an efficient control 
law so that the control system presents the desired performance. A practical approach for 
controllers design is from the mathematical model of the plant to be controlled. 

Mathematical modeling is a set of heuristic and /or computational procedures properly 
established on a real plant in order to obtain a mathematical equation (models) to represent 
accurately its dynamic behavior in operation. There are three basic approaches for 
mathematical modeling: 

• White box modeling. In this case, such models can be satisfactorily obtained from 
the physical laws governing the dynamic behavior of the plant. However, this may be 
a limiting factor in practice, considering plants with uncertainties, nonlinearities, time 
delay, parametric variations, among other dynamic complexity characteristics. The poor 
understanding of physical phenomena that govern the plant behavior and the resulting 
model complexity, makes the white box approach a difficult and time consuming task. 
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In addition, a complete understanding of the physical behavior of a real plant is almost 
impossible in many practical applications. 

• Black box modeling. In this case, if such models, from the physical laws, are difficult 
or even impossible to obtain, is necessary the task of extracting a model from experimental 
data related to dynamic behavior of the plant. The modeling problem consists in choosing 
an appropriate structure for the model, so that enough information about the dynamic 
behavior of the plant can be extracted efficiently from the experimental data. Once the 
structure was determined, there is the parameters estimation problem so that a quadratic 
cost function of the approximation error between the outputs of the plant and the model 
is minimized. This problem is known as systems identification and several techniques 
have been proposed for linear and nonlinear plant modeling. A limitation of this approach 
is that the structure and parameters of the obtained models usually do not have physical 
meaning and they are not associated to physical variables of the plant. 

• Gray box modeling. In this case some information on the dynamic behavior of the 
plant is available, but the model structure and parameters must be determined from 
experimental data. This approach, also known as hybrid modeling, combines the features 
of the white box and black box approaches. 

The area of mathematical modeling covers topics from linear regression up to sofisticated 
concepts related to qualitative information from expert, and great attention have been given 
to this issue in the academy and industry (Abonyi et al., 2000; Brown & Harris, 1994; Pedrycz 
& Gomide, 1998; Wang, 1996). A mathematical model can be used for: 

• Analysis and better understanding of phenomena (models in engineering, economics, 
biology, sociology, physics and chemistry); 

• Estimate quantities from indirect measurements, where no sensor is available; 

• Hypothesis testing (fault diagnostics, medical diagnostics and quality control); 

• Teaching through simulators for aircraft, plants in the area of nuclear energy and patients 
in critical conditions of health; 

• Prediction of behavior (adaptive control of time-varying plants); 

• Control and regulation around some operating point, optimal control and robust control; 

• Signal processing (cancellation of noise, filtering and interpolation); 

Modeling techniques are widely used in the control systems design, and successful 
applications have appeared over the past two decades. There are cases in which the 
identification procedure is implemented in real time as part of the controller design. This 
technique, known as adaptive control, is suitable for nonlinear and/or time varying plants. In 
adaptive control schemes, the plant model, valid in several operating conditions is identified 
on-line. The controller is designed in accordance to current identified model, in order to 
garantee the performance specifications. There is a vast literature on modeling and control 
design (Astrom & Wittenmark, 1995; Keesman, 2011; Sastry & Bodson, 1989; Isermann & 
Miinchhof, 2011; Zhu, 2011; Chalam, 1987; Ioannou, 1996; Lewis & Syrmos, 1995; Ljung, 1999; 
Soderstrom & Stoica, 1989; Van Overschee & De Moor, 1996; Walter & Pronzato, 1997). Most 
approaches have a focus on models and controllers described by linear differential or finite 
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differences equations, based on transfer functions or state space representation. Moreover, 
motivated by the fact that all plant present some type of nonlinear behavior, there are several 
approaches to analysis, modeling and control of nonlinear plants (Tee et al., 2011; Isidori, 
1995; Khalil, 2002; Sjoberg et al., 1995; Ogunfunmi, 2007; Vidyasagar, 2002), and one of the 
key elements for these applications are the fuzzy systems (Lee et al., 2011; Hellendoorn 
& Driankov, 1997; Grigorie, 2010; Vukadinovic, 2011; Michels, 2006; Serra & Ferreira, 2011; 
Nelles, 2011). 

2. Fuzzy inference systems 

The theory of fuzzy systems has been proposed by Lotfi A. Zadeh (Zadeh, 1965; 1973), as 
a way of processing vague, imprecise or linguistic information, and since 1970 presents 
wide industrial application. This theory provides the basis for knowledge representation 
and developing the essential mechanisms to infer decisions about appropriate actions to be 
taken on a real problem. Fuzzy inference systems are typical examples of techniques that 
make use of human knowledge and deductive process. Its structure allows the mathematical 
modeling of a large class of dynamical behavior, in many applications, and provides greater 
flexibility in designing high-performance control with a certain degree of transparency for 
interpretation and analysis, that is, they can be used to explain solutions or be built from 
expert knowledge in a particular field of interest. For example, although it does not know 
the exact mathematical model of an oven, one can describe their behavior as follows: " IF 
is applied more power on the heater THEN the temperature increases", where more and 
increases are linguistic terms that, while imprecise, they are important information about 
the behavior of the oven. In fact, for many control problems, an expert can determine a 
set of efficient control rules based on linguistic descriptions of the plant to be controlled. 
Mathematical models can not incorporate the traditional linguistic descriptions directly into 
their formulations. Fuzzy inference systems are powerful tools to achieve this goal, since 
the logical structure of its IF < antecedent proposition> THEN < consequent proposition> 
rules facilitates the understanding and analysis of the problem in question. According to 
consequent proposition, there are two types of fuzzy inference systems: 

• Mamdcmi Fuzzy Inference Systems: In this type of fuzzy inference system, the antecedent and 
consequent propositions are linguistic informations. 

• Takagi-Sugeno Fuzzy Inference Systems: In this type of fuzzy inference system, the antecedent 
proposition is a linguistic information and the consequent proposition is a functional 
expression of the linguistic variables defined in the antecedent proposition. 

2.1 Mamdani fuzzy inference systems 

The Mamdani fuzzy inference system was proposed by E. H. Mamdani (Mamdani, 1977) to 
capture the qualitative knowledge available in a given application. Without loss of generality, 
this inference system presents a set of rules of the form: 

9t' : IF x x is FL AND . . . AND x„ is FL THEN y is GL (1) 

j\ x \ ]\ x n u ]\y 

In each rule i |i t=1 ' 2 '— ''J, w here / is the number of rules, x.\,xi, ■ ■ ■ ,x n are the linguistic 
variables of the antecedent (input) and y is the linguistic variable of the consequent (output), 
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defined, respectively, in the own universe of discourse 1A% X , . . . ,Ux„ e IV- The fuzzy sets 
F?r_ ,F;r. ,...,F,L e G',-, are the linguistic values (terms) used to partition the unierse of 
discourse of the linguistic variables of antecedent and consequent in the inference system, 

that is, ^ € {%,F^ F^}^ "andG^ 6 {0^,^,, G^}, where p s , 

and p,y are the partitions number of the universes of discourses associated to the linguistic 
variables Xt and y, respectively. The variable X % belongs to the i\i7.zy set Fl - with a value }ip, _ 
defined by the membership function ]i\ : R — > [0, 1], where ]i\, € {}ip , ]i\ _ , ■ ■ ■ , ]i\ }. 
The variable y belongs to the fuzzy set G',~ with a value \i 1 q defined by the membership 
function yL : R — > [0, 1] where fi' G € {}i'q ,^' g ,...,ji' G }. Each rule is interpreted by a 

fuzzy implication 

9t' : ui -kill -k ... -k iL -> w' r (2) 

where * is a T-norm, ^p_ *?*f . * . . . * ytp is the fuzzy relation between the linguistic inputs, 
on the universes of discourses Wj, x U% 2 x . . . x Wj„, and fig. is the linguistic output defined 
on the universe of discourse y. The Mamdani inference systems can represent MISO (Multiple 
Input and Single Output) systems directly, and the set of implications correspond to a unique 
fuzzy relation in Ux x x Wj 2 x . . . x Ux„ X y of the form 

/ 

^MISO ■ V b'F jlh * VV AH *■■■* V F, |Il; * Pg ; J ( 3 ) 

1=1 

where \j is a S-norm. 

The fuzzy output m | L m=1 A-v''J j s given by 

G(y m ) = JH M /so ° (f L. * f L, * • ■ • * f F, f , ) ( 4 ) 

71*1 /l A 2 i\ x n 

where o is a inference based composition operator, which can be of the type max-min or 
max-prodnct, and xf is any point in U Xt . The Mamdani inference systems can represent MIMO 
(Multiple Input and Multple Output) systems of r outputs by a set of r MISO sub-rules coupled 
base & mso | \J= l *-J\, that is, 



G(y) = ^mimo ° (?4, f , * Ff,- * ■ • • * Ff,,.. ) ( 5 ) 



with G(y) = [G(g 1 ),...,G(y r )] T and 

r / 



W-MIMO ■ U i V Mfy*, * ^F, 1i2 * • ■ • * f'%,, * ^G /Ll? „, 1 1 ( 6 ) 



m=l i=l 



where the operator [J represents the set of all fuzzy relations JH^JSO associated to each output 
ym- 
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2.2 Takagi-Sugeno fuzzy inference systems 

The Takagi-Sugeno fuzzy inference system uses in the consequent proposition, a functional 
expression of the linguistic variables defined in the antecedent proposition (Takagi & Sugeno, 
1985). Without loss of generality the i \ '■'^ ' '"' >-th rule of this inference system, where / is the 
maximum number of rules, is given by: 

R' : IF % is FL AND . . . AND x n is F!,. THEN ft = f,(x) (7) 

The vector x g 5R" contains the linguistic variables of the antecedent proposition. Each 
linguistic variable has its own universe of discourse Ufa , ■ ■ ■ , U% n partitioned by fuzzy sets 
which represent the linguistic terms. The variable Xt l'- 1 - 2 -- ■<» belongs to the fuzzy set 
F-i- with value y,\ defined by a membership function \i\ : R — ¥ [0,1], with jtV £ 

{ftp ,jij: ,...,jip }, where p% t is the partitions number of the universe of discourse 

associated to the linguistic variable X f. The activation degree hj of the rule i is given by: 

k<(*) = V? iK * V-v^ * • ■ • * V? m ( 8 ) 

where x[ is any point in lAg r The normalized activation degree of the rule i is defined as: 

L r= ih r (x) 
This normalization implies that 

/ 

E?,(*) = i (io) 

The response of the Takagi-Sugeno fuzzy inference system is a weighted sum of the functional 
expressions defined on the consequent proposition of each rule, that is, a convex combination 
of local functions /,: 

iJ=El,(x)f l (x) (11) 

1=1 

Such inference system can be seen as linear parameter varying system. In this sense, the 
Takagi-Sugeno fuzzy inference system can be considered as a mapping from antecedent space 
(input) to the convex region (polytope) defined on the local functional expressions in the 
consequent space. This property allows the analysis of the Takagi-Sugeno fuzzy inference 
system as a robust system which can be applied in modeling and controllers design for 
complex plants. 

3. Fuzzy computational modeling based control 

Many human skills are learned from examples. Therefore, it is natural establish this "didactic 
principle" in a computer program, so that it can learn how to provide the desired output as 
function of a given input. The Computational intelligence techniques, basically derived from 
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the theory of Fuzzy Systems, associated to computer programs, are able to process numerical 
data and/or linguistic information, whose parameters can be adjusted from examples. The 
examples represent what these systems should respond when subjected to a particular input. 
These techniques use a numeric representation of knowledge, demonstrate adaptability and 
fault tolerance in contrast to the classical theory of artificial intelligence that uses symbolic 
representation of knowledge. The human knowledge, in turn, can be classified into two 
categories: 

1. Objective knowledge: This kind of knowledge is used in the engineering problems 
formulation and is defined by mathematical equations (mathematical model of a 
submarine, aircraft or robot; statistics analysis of the communication channel behaviour; 
Newton's laws for motion analysis and Kirchhoff 's Laws for circuit analysis). 

2. Subjective knowledge: This kind of knowledge represents the linguistic informations defined 
through set of rules, knowledge from expert and design specifications, which are usually 
impossible to be described quantitatively. 

Fuzzy systems are able to coordinate both types of knowledge to solve real problems. The 
necessity of expert and engineers to deal with increasingly complex control systems problems, 
has enabled via computational intelligence techniques, the identification and control of real 
plants with difficult mathematical modeling. The computational intelligence techniques, 
once related to classical and modern control techniques, allow the use of constraints in 
its formulation and satisfaction of robustness and stability requirements in an efficient and 
practical form. The implementation of intelligent systems, especially from 70's, has been 
characterized by the growing need to improve the efficiency of industrial control systems in 
the following aspects: increasing product quality, reduced losses, and other factors related to 
the improvement of the disabilities of the identification and control methods. The intelligent 
identification and control methodologies are based on techniques motivated by biological 
systems, human intelligence, and have been introduced exploring alternative representations 
schemes from the natural language, rules, semantic networks or qualitative models. 

The research on fuzzy inference systems has been developed in two main directions. The first 
direction is the linguistic or qualitative information, in which the fuzzy inference system is 
developed from a collection of rules (propositions). The second direction is the quantitative 
information and is related to the theory of classical and modern systems. The combination 
of the qualitative and quantitative informations, which is the main motivation for the use 
of intelligent systems, has resulted in several contributions on stability and robustness of 
advanced control systems. In (Ding, 2011) is addressed the output feedback predictive control 
for a fuzzy system with bounded noise. The controller optimizes an infinite-horizon objective 
function respecting the input and state constraints. The control law is parameterized as a 
dynamic output feedback that is dependent on the membership functions, and the closed-loop 
stability is specified by the notion of quadratic boundedness. In (Wang et al., 2011) is 
considered the problem of fuzzy control design for a class of nonlinear distributed parameter 
systems that is described by first-order hyperbolic partial differential equations (PDEs), where 
the control actuators are continuously distributed in space. The goal of this methodology is to 
develop a fuzzy state-feedback control design methodology for these systems by employing 
a combination of PDE theory and concepts from Takagi-Sugeno fuzzy control. First, the 
Takagi-Sugeno fuzzy hyperbolic PDE model is proposed to accurately represent the nonlinear 
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first-order hyperbolic PDE system. Subsequently, based on the Takagi-Sugeno fuzzy-PDE 
model, a Lyapunov technique is used to analyze the closed-loop exponential stability with a 
given decay rate. Then, a fuzzy state-feedback control design procedure is developed in terms 
of a set of spatial differential linear matrix inequalities (SDLMIs) from the resulting stability 
conditions. The developed design methodology is successfully applied to the control of a 
nonisothermal plug-flow reactor. In (Sadeghian & Fatehi, 2011) is used a nonlinear system 
identification method to predict and detect process fault of a cement rotary kiln from the 
White Saveh Cement Company. After selecting proper inputs and output, an inputUoutput 
locally linear neuro-fuzzy (LLNF) model is identified for the plant in various operation points 
in the kiln. In (Li & Lee, 2011) an observer-based adaptive controller is developed from a 
hierarchical fuzzy-neural network (HFNN) is employed to solve the controller time-delay 
problem for a class of multi-input multi-output(MIMO) non-affine nonlinear systems under 
the constraint that only system outputs are available for measurement. By using the implicit 
function theorem and Taylor series expansion, the observer-based control law and the weight 
update law of the HFNN adaptive controller are derived. According to the design of the 
HFNN hierarchical fuzzy-neural network, the observer-based adaptive controller can alleviate 
the online computation burden and can guarantee that all signals involved are bounded and 
that the outputs of the closed-loop system track asymptotically the desired output trajectories. 

Fuzzy inference systems are widely found in the following areas: Control Applications 
- aircraft (Rockwell Corp.), cement industry and motor/valve control (Asea Brown 
Boveri Ltd.), water treatment and robots control (Fuji Electric), subway system (Hitachi), 
board control (Nissan), washing machines (Matsushita, Hitachi), air conditioning system 
(Mitsubishi); Medical Technology - cancer diagnosis (Kawasaki medical School); Modeling 
and Optimization - prediction system for earthquakes recognition (Institute of Seismology 
Bureau of Metrology, Japan); Signal Processing For Adjustment and Interpretation - 
vibration compensation in video camera (Matsushita), video image stabilization (Matsushita 
/ Panasonic), object and voice recognition (CSK, Hitachi Hosa Univ., Ricoh), adjustment of 
images on TV (Sony). Due to the development, the many practical possibilities and the 
commercial success of their applications, the theory of fuzzy systems have a wide acceptance 
in academic community as well as industrial applications for modeling and advanced control 
systems design. 

4. Takagi-Sugeno fuzzy black box modeling 

This section aims to illustrate the problem of black box modeling, well known as systems 
identification, addressing the use of Takagi-Sugeno fuzzy inference systems. The nonlinear 
input-output representation is often used for building TS fuzzy models from data, where the 
regression vector is represented by a finite number of past inputs and outputs of the system. 
In this work, the nonlinear autoregressive with exogenous input (NARX) structure model is 
used. This model is applied in most nonlinear identification methods such as neural networks, 
radial basis functions, cerebellar model articulation controller (CMAC), and also fuzzy logic. 
The NARX model establishes a relation between the collection of past scalar input-output data 
and the predicted output 

2/Jc+i = F[y k , . . .,y k _ n+v u k ,...,..., u k _ lhi+l ] (12) 
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where k denotes discrete time samples, n-y and n u are integers related to the system's order. In 
terms of rules, the model is given by 

R' : IF y k is F[ AND ■ ■ ■ AND y k _ n +1 is F l n AND u k is G[ AND ■ ■ ■ AND u k _„ u+1 is G'„ u 
THEN y\ +l = £ a i,jyk-j+\ + £ hj u k-j+l + a (13) 

where a,,-, b; ,• and c,- are the consequent parameters to be determined. The inference formula 
of the TS fuzzy model is a straightforward extension of (11) and is given by 

/ 
2/Jc+i = ^ (14) 

! = 1 



/ 

»+i = E^wii (1 5 ) 

(=1 

with 

35 = fc • • • / yjc-ny+1/ "/c- • ■ • / «(c-«„+l] (16) 

and hj(x) is given as (8). This NARX model represents multiple input and single output 
(MISO) systems directly and multiple input and multiple output (MIMO) systems in a 
decomposed form as a set of coupled MISO models. 

4.1 Antecedent parameters estimation problem 

The experimental data based antecedent parameters estimation can be done by fuzzy clustring 
algorithms. A cluster is a group of similar objects. The term "similarity" should be understood 
as mathematical similarity measured in some well-define sense. In metric spaces, similarity 
is often defined by means of a distance norm. Distance can be measured from data vector to 
some cluster prototypical (center). Data can reveal clusters of different geometric shapes, sizes 
and densities. The clusters also can be characterized as linear and nonlinear subspaces of the 
data space. 

The objective of clustering is partitioning the data set Z into c clusters. Assume that c is 
known, based on priori knowledge. The fuzzy partition of Z can be defined as a family of 
subsets {A, |1 < i < c} C P(Z), with the following properties: 

U A = Z (17) 

1=1 

A { n A, ■ = (18) 
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C A, ■ C Z, (19) 

Equation (17) means that the subsets A, collectively contain all the data in Z. The subsets 
must be disjoint, as stated by (18), and none off them is empty nor contains all the data in Z, 
as stated by (19). In terms of membership functions, ]l&. is the membership function of A,-. To 
simplifly the notation, in this paper is used \i ik instead of zz,- (zj.). The c x N matrix U = \}ift\ 
represents a fuzzy partitioning space if and only if: 

M fc = J U e $t cxN \p lk e [0,l],Vi,fc £ M = l,Vfc;0 < £ Vlk < N,Wi 1 (20) 

I i=l k=l J 

The z'-th row of the fuzzy partition matrix U contains values of the z-th membership function 
of the fuzzy subset A, of Z. The clustering algorithm optimizes an initial set of centroids by 
minimizing a cost function / in an iterative process. This function is usually formulated as: 

c N 
J(Z;U,V,A) = £E^dL, (21) 

i=lk=l 

where, Z = \z\,Zii ' ' ' < z n} is a finite data set. U = [fife] € Mr c is a fuzzy partition of Z. 
V = {v^, V2, ■ ■ ■ , v c } , v; 6 5R", is a vector of cluster prototypes (centers). A denote a c-tuple of 
the norm-induting matrices: A = (A\, Ai, ■ ■ ■ , A c ). D ikA is a square inner-product distance 
norm. The m G [1, oo) is a weighting exponent which determines the fuzziness of the clusters. 
The clustering algorithms differ in the choice of the norm distance. The norm metric influences 
the clustering criterion by changing the measure of dissimilarity. The Euclidean norm induces 
hiperspherical clusters. It's characterizes the FCM algorithm, where the norm-inducing matrix 
A; is equal to identity matrix {A; = I), which strictly imposes a circular shape to all 
clusters. The Euclidean Norm is given by: 

D LM = ( z 'c-^) T A !fCM ( Z(t -^) (22) 

An adaptative distance norm in order to detect clusters of different geometrical shapes in a 
data set characterizes the GK algorithm: 

D l CK = ( z k-Vi) r A iGK {z k -Vi) (23) 

In this algorithm, each cluster has its own norm-inducing matrix A; , where each cluster 
adapts the distance norm to the local topological structure of the data set. A; is given by: 

A i<x =\p i det(F i )] 1/n Fr\ (24) 

where p, is cluster volume, usually fixed in 1. The n is data dimension. The Fj is fuzzy 
covariance matrix of the z-th cluster defined by: 

N 

I] (Hik) m (zk-Vi)(,z k -Vi) T 
Fi = — s (25) 

E (?<*)'" 

k=\ 
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The eigenstructure of the cluster covariance matrix provides information about the shape and 
orientation cluster. The ratio of the hyperellipsoid axes is given by the ratio of the square 
roots of the eigenvalues of F,. The directions of the axes are given by the eigenvectores of 
F,. The eigenvector corresponding to the smallest eigenvalue determines the normal to the 
hyperplane, and it can be used to compute optimal local linear models from the covariance 
matrix. The fuzzy maximum likelihood estimates (FLME) algorithm employs a distance norm 
based on maximum lekelihood estimates: 



V G iFLME 
D *FLME = 5 eX P 



^fa-VifF^fa-Vi) 



(26) 



Note that, contrary to the GK algorithm, this distance norm involves an exponential term and 
decreases faster than the inner-product norm. The Fi FhuE denotes the fuzzy covariance matrix 
of the ;-th cluster, given by (25). When m is equal to 1, it has a strict algorithm FLME. If m 
is greater than 1, it has a extended algorithm FLME, or Gath-Geva (GG) algorithm. Gath 
and Geva reported that the FLME algorithm is able to detect clusters of varying shapes, 
sizes and densities. This is because the cluster covariance matrix is used in conjuncion with 
an "exponential" distance, and the clusters are not constrained in volume. P; is the prior 
probability of selecting cluster i, given by: 

P i = l L M" ( 27 ) 

/c=l 

4.2 Consequent parameters estimation problem 

The inference formula of the TS fuzzy model in (15) can be expressed as 

yjfc+1 = 7lOjfc)[«l,iy* + • • ■ + «l,nyyjt-By+l + b l,l"/c + • ■ • + h,nu^k-n u +l + c l] + 
l2{x k )[a 2 ,iyk + ■■■ + a 2 ,,n J yk-n l ,+\ + h 2,l u k + ■■■ + h,nu u k-n u +\ + c l\ + 



7l{x k ) [a;,iy* + • ■ • + fl/,nyyjc-«j,+l + h l,l u k +■■■ + bl,nuUk-n u +l + Cl] (28) 

which is linear in the consequent parameters: a, b and c. For a set of N input-output data 
pairs { (asfc, yj-)|i = 1,2, . . . , N} available, the following vetorial form is obtained 

Y =[il> 1 X,i> 2 X,...,il>iX]e + 3 (29) 

where tp { = diag(yi{x k )) e K NxN , X = [y k , . . . , y k - ny +i, u k , . . . , u k _ nu+1 , 1] G 
!R Nx( "' + "" +1 ), Y e K Nxl , E G K Nxl and 6 G sft'K+^+i)*! are the normalized membership 
degree matrix of (9), the data matrix, the output vector, the approximation error vector and 
the estimated parameters vector, respectively. If the unknown parameters associated variables 
are exactly known quantities, then the least squares method can be used efficiently. However, 
in practice, and in the present context, the elements of X are no exactly known quantities so 
that its value can be expressed as 

Vk = x[0 + n k (30) 



Highlighted Aspects from Black Box Fuzzy Modeling for Advanced Control Systems Design 1 1 

where, at the Jc-th sampling instant, x[ = [l\(. x k + $k)' ■ ■ ■ > Tk( x k + &)] is the vector of the 
data with error in variables, x^ = [i/n, . • • , l/fc-n, / M /c-l/ • • •/ u k-n„' 1] is the vector of the 
data with exactly known quantities, e.g., free noise input-output data, £/ c is a vector of noise 
associated with the observation of ajj-, and rn- is a disturbance noise. 

The normal equations are formulated as 

[t,xjxj]6k=t,w (3i) 

7=1 7=1 

and multiplying by - gives 
K 

{ l E [rj(*j + €y) 7J("7 + €;)] bjO" 5 ; + €/) 7J(*; + £ y )] T }0/c = 



/=i 



Noting that y y = x j& + 1j, 



1 



k 



7=1 



(JEbi^ + y ?;(^ + ^)][?)(^ + ^) y'j(xj + Zj)] T }d k 



7=1 7 =1 

€;•),..., 7J(*; + €/)]'/,• 03) 
and 

«t = {rE[7y («; + €/) 7J(*y + €;)][7}(*; + €;) 7J(^ + ^)] T }-^E[7/(^'+ 

y=l ;=1 

€/) TJ(*; + €;)]7/34) 

where 0^ = 0^ — is the parameter error. Taking the probability in the limit as k — > oo, 



K1' 



p.lim k = p.lim {^q- 1 ^} (35) 



with 

C k = £ [7}(ay + $), • • • , 7J(^ + £;)] [7/ (»/ + €j), • • ., 7J(*j + iy)] 1 

7=1 

b/c = E [7/ (»; + €/)/ • • •< 7J(«; + £,)]'/, 
7=1 
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Applying Slutsky's theorem and assuming that the elements of yC k and \b k converge in 
probability, we have 

p.lim 8 k = P-lim yC^ p.lim -b k (36) 

Thus, 

k 
k 



1 1 k 

p.lim -C k =p.lim -r E hj(ay + £/)/ ■ • ■ - 7/(*J + «;)] hjtej + £/)> ■ • ■ - 7J(*/ + f/)] 1 



;'=i 



11* 1 *" 

p.lim -C t =p.lim - E(7 / 1 ) 2 (^ + «/)(*> + £jf + • . • + P-lim - J^W^i + tj)(xj + fy) 



Assuming x; and £,■ statistically independent, 

£ h)) 2 [x,x] + ttf] + ... + p.lim i £ 

7 =1 7=1 



11 1 

p.lim -C k ^.lim - ^( 7 ]) 2 [x jX j + Sfij] + ... + p.lim ^ ^(ij) 2 !^ + ^ ; Tl 



p.lim -C k =p.lim - £ ^/ [(7, 1 ) 2 + • • • + I?)) 2 ] + P-lim - £ Cy«f [(7, 1 ) 2 + • • • + (?;) 2 ](37) 

;=1 ;=1 

1 
with Yj 7; = 1. Hence, the asymptotic analysis of the TS fuzzy model consequent parameters 

;=i 
estimation is based in a weighted sum of the fuzzy co variance matrices of x and £. Similarly, 

1 1 * 

p.lim -b k = p.lim - £ [jj(xj + £j), ..., yfaj + $j))t]j 

1 1 k 

p.lim -b k = p.lim - £ [tjtyj, ■■■, JjZjtlj] (38) 

Substituting from (37) and (38) in (36), results 

p.lim 6 k = {p.lim \ £ xyxJ[( 7 J) 2 + • - - + (?j) 2 ] + P-Urn 1 £ t$Un}? + ■ • ■ 
7=1 7 =1 

+ ( 7 )) 2 ] } -V-iim \ t bkpij 7kw} 09) 

7=1 

! 
with Y^ yl = 1. For the case of only one rule (/ = 1), the analysis is simplified to the linear one, 

i=l 
with 7 J |rt fc = 1. Thus, this analysis, which is a contribution of this article, is an extension of 
the standard linear one, from which can result several studies for fuzzy filtering and modeling 
in a noisy environment, fuzzy signal enhancement in communication channel, and so forth. 
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Provided that the input u k continues to excite the process and, at the same time, the coefficients 
in the submodels from the consequent are not all zero, then the output y k will exist for all k 

k 
observation intervals. As a result, the fuzzy covariance matrix YJ x i x j [(?!) + ■ • ■ + (7;) ] 

7=1 
will also be non-singular and its inverse will exist. Thus, the only way in which the asymptotic 

error can be zero is for £,f/, identically zero. But, in general, £,- and n; are correlated, the 

asymptotic error will not be zero and the least squares estimates will be asymptotically biased 

to an extent determined by the relative ratio of noise to signal variances. In other words, least 

squares method is not appropriate to estimate the TS fuzzy model consequent parameters in 

a noisy environment because the estimates will be inconsistent and the bias error will remain 

no matter how much data can be used in the estimation. 

As a consequence of this analysis, the definition of the vector [/3 z,, . . .,/3z,] as fuzzy 
instrumental variable vector or simply the fuzzy instrumental variable (FIV) is proposed. Clearly, 
with the use of the FIV vector in the form suggested, becomes possible to eliminate the 
asymptotic bias while preserving the existence of a solution. However, the statistical 
efficiency of the solution is dependent on the degree of correlation between [f>- z;, . . . , fazj] 

and Mscy, . . . , 7,-ai;]. In particular, the lowest variance estimates obtained from this approach 

occur only when z, = x; and f>'- | '.",' '"' k = j\ | ti'"7 / i- e -/ when the z, are equal to the dynamic 
system "free noise" variables, which are unavailable in practice. According to situation, 
several fuzzy instrumental variables can be chosen. An effective choice of FIV would be the 
one based on the delayed input sequence 



[u k _ T , ..., u k _ T _ n , u k ,..., u k _ n 



T 



where T is chosen so that the elements of the fuzzy covariance matrix C zx are maximized. In 
this case, the input signal is considered persistently exciting, e.g., it continuously perturbs or 
excites the system. Another FIV would be the one based on the delayed input-output sequence 

z j - [))k-l-dir ■ ■ >yk-n y -dl> u k-l-dl>- ' ' > u k-n„-dl\ 

where dl is the applied delay. Other FIV could be the one based in the input-output from 
a "fuzzy auxiliar model" with the same structure of the one used to identify the nonlinear 
dynamic system. Thus, 



\9k-V ' ' >yk-n„> u k-l> • ■ ■ , u k-n„ 



T 



where y k is the output of the fuzzy auxiliar model, and u k is the input of the dynamic system. 
The inference formula of this fuzzy auxiliar model is given by 

Vk+\ = fa ( z k) WuVk + ■■■ + <*l,ny$k-ny+l + Pl,l u k + ■■■ + Pl,n U Uk-n u +l + h\ + 
fa(zk) [<*2,iyk + ■■■ + 0C2,nyfk-ny+l + Pl,l u k + ■■■ + P2,nuUk-n u +l + fa] + 



3/ (*jfc) Wl.lVk + ■■■ + ^,ni,yk-n y +l + Pl,\u k + ... + p\, nu u k _ nu+x + 6,} (40) 
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which is also linear in the consequent parameters: ot, p and S. The closer these parameters are 
to the actual, but unknown, system parameters (a, b, c), more correlated zj- and iej- will be, 
and the obtained FIV estimates closer to the optimum. 

4.2.1 Batch processing scheme 

The normal equations are formulated as 

k k 

£[$*j /3j*;]hj(»/ + fy) l){*j + Zi)] T h- £[p)*j P)zj\yj = o (41) 

i=l 7=1 

or, with fy =[/$}*;,..., $z ; -], 

[ECjxf]9k-EtM = o (42) 

7=1 ;=1 

so that the FIV estimate is obtained as 

k k 

h = &} z i P)*j\[yfa} + Sj) faj + Sj)?}- 1 ^)*} fajtoj (43) 

/=1 7=1 

and, in vectorial form, the interest problem may be placed as 

= (r T E)" 1 r T r (44) 

where T S sjj'(n y +n„+l)xN j g ^ e f UZZ y extended instrumental variable matrix with rows 
given by £,, E G Sft H"y+"»+l) j s the fuzzy extended data matrix with rows given by pfj and 
F G sftNxl ig the out p ut vec tor and 9 G sftK"j+"«+l)xl [ s the parameters vector. The models 
can be obtained by the following two approaches: 

• Global approach : In this approach all linear consequent parameters are estimated 
simultaneously, minimizing the criterion: 

6 = arg min || Y T L6 - T T Y \\j (45) 

• Local approach : In this approach the consequent parameters are estimated for each rule i, 
and hence independently of each other, minimizing a set of weighted local criteria (i = 
1,2,...,/): 

§i = arg min || Z T ^ l XQ l - Z T T,y || 2 (46) 

where Z has rows given by z, and Y; is the normalized membership degree diagonal 
matrix according to zs. 

Example 1. So that the readers can understand the definitions of global and local fuzzy 
modeling estimations, consider the following second-order polynomial given by 

y = lu\ - 4m, c + 3 (47) 
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where u k is the input and y k is the output, respectively. The TS fuzzy model used to 
approximate this polynomial has the following structure with 2 rules: 

R l : IF Mjt is F, THEN yjt = fl + a\u k 

where i = 1,2. It was choosen the points u k = —0.5 and u k = 0.5 to analysis the consequent 
models obtained by global and local estimation, and it was defined triangular membership 
functions for —0.5 < U^ < 0.5 in the antecedent. The following rules were obtained: 



Local estimation: 



Global estimation: 



R 1 : IF u k is - 0.5 THEN y = 3.1000 - 4.4012u t 
R 2 : IF u k is + 0.5 THEN y = 3.1000 - 3.5988k,; 



R 1 : IF u k is - 0.5 THEN y = 4.6051 - 1.7503m* 
R 2 : IF u k is + 0.5 THEN y = 1.3464 + 0.3807w t 

The application of local and global estimation to the TS fuzzy model results in the consequent 
models given in Fig. 1. The consequent models obtained by local estimation describe properly 
the local behavior of the function and the fuzzy model can easily be interpreted in terms of the 
local behavior (the rule consequents). The consequent models obtained by global estimation 
are not relevant for the local behavior of the nonlinear function. The fit of the function is 









. 


^^^^ y k = 4.6051 


-1.7503u k 




- 






y k 


= 1 .3464 + 0.3807u k 








- 







-0.5 



0.5 




Fig. 1. The nonlinear function and the result of global (top) and local (bottom) estimation of 
the consequent parameters of the TS fuzzy models. 

shown in Fig. 2. The global estimation gives a good fit and a minimal prediction error, but 
it bias the estimates of the consequent as parameters of local models. In the local estimation 
a larger prediction error is obtained than with global estimation, but it gives locally relevant 
parameters of the consequent. This is the tradeoff between local and global estimation. All 
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the results of the Example 1 can be extended for any nonlinear estimation problem and they 
would be considered for computational and experimental results analysis in this paper. 




nonlinear function 
global estimation 



Fig. 2. The nonlinear function approximation result by global (top) and local (bottom) 
estimation of the consequent parameters of the TS fuzzy models. 



4.2.2 Recursive processing scheme 

An on line FIV scheme can be obtained by utilizing the recursive solution to the FIV equations 
and then updating the fuzzy auxiliar model continuously on the basis of these recursive 
consequent parameters estimates. The FIV estimate in (43) can take the form 



0k = Ptb 



m 



where 



and 



7=1 



r-i-i 



7=1 
which can be expressed as 

p k l = P k-i + \fik*b ■■■- fa] b) (*k + &), • • • , 7t(»* + Zk)] 1 
and 



h = h-i + [PW, ■ ■ ■ , P'k z k}yk 



(48) 



(49) 
(50) 
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respectively. Pre-multiplying (49) by P k and post-multiplying by P k -\ gives 

A-i = A + PM*h ■ • •,$**] [?/ (as* + &) Ti(asjfc + ?Jc)] T A-i (51) 

then post-multiplying (51) by the FIV vector [fyjZj, . . . , fiiZj], results 

Pi_a [j8j« fo . . . , j8}*jt] = A [$** jSJzjfc] + P k {Pbk jBJzjJ [7; 1 (xj + &),... , 7Jt(**+ 

&)] T A-l[/5j*b...//5}«lt](52) 
A-ife ■ • .,/3J*it] = Afe' • •■/J8jzj fc ]{l + hjfo + &), • ..,7t(** + &)fA-l 

[^ fo ...,j8}zjfc]}(53) 
Then, post-multiplying by 

{l+[7 ; 1 (^ + ^),...,7[(^ + ^)] T A-ife,...,/3;^]}- 1 [7 ; 1 (^ + ^),---, 

7k(*k + Zk)] T Pk-l (54) 
we obtain 

P w fe,...,/J| zt ]{l + [ 7 j(x k + &), . ..,7Jt(** + ^)] T -F)t-lfe---^H]}" 1 

[7-(^ + ft), • • •> 7*(»* + ^)] T A-i = 
P k \^ k z k ,...,^z k ]{ 7 }{x k + Z k ),..., 7 [(x k + ^)] T A-1 (55) 
Substituting (51) in (55), we have 

P k = Ph - Pn[^ t , . ..,p)z k ]{l + [ 7 }{x k + fo), . .., 7 [(x k + &)] T Pn 

[j5lz k ,...,li' l z k }}- 1 { 7 j(x k + ^) / ..., 7 ' k (x k + ^)} T P k _ 1 (56) 

Substituting (56) and (50) in (48), the recursive consequent parameters estimates will be: 

h = { Pk-i - Pk-\\fiW • • ./^*jt]{i + [-r}(*k + Ski ■ ■■,-y k { x k + Zk)] T Pk-i 

\f,\z k , . ..,$\z k \}-\ 7 ){x k + &), . .., 7 {(x k + m T Pk-i}{b k -i + [fiz kf . ..,p> k z k ]y k } (57) 
so that finally, 

§ k = 6 k _ x - K k { [ 7 ){x k + &), . . . , 7 [{x k + &)] T 4fc-l - y/d (58) 

where 

K k = P M \$\z k f,\z k \{\ + [ 7 j(x k + &), . . . , y fc (« t + &)] r iVl[$*jfc, . . . rj S}^ fe ]>- 1 (59) 

Equations (56)-(59) compose the recursive algorithm to be implemented so the consequent 
parameters of a Takagi-Sugeno fuzzy model can be estimated from experimental data. 
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5. Results 

In the sequel, some results will be presented to demonstrate the effectiveness of black box 
fuzzy modeling for advanced control systems design. 

5.1 Computational results 

5.1.1 Stochastic nonlinear SISO system identification 

The plant to be identified consists on a second order highly nonlinear discrete-time system 

VkVk-iiVk + 2-5) 

Vk+l = — — t~, 2 + U k + e k ( 60 ) 

which is a benchmark problem in neural and fuzzy modeling, where j/; c is the output and 
Mj- = sin( =z£- ) is the applied input. In this case e k is a white noise with zero mean and variance 
a . The TS model has two inputs y k and i/jt-1 an( A a single output i/fc+l/ an d the antecedent 
part of the fuzzy model (the fuzzy sets) is designed based on the evolving clustering method 
(ECM). The model is composed of rules of the form: 

R' : IF y k is F{ AND y k _ t is Fj THEN 

y'k+i = a i,iVk + a i,iyk-\ + b i,i u k + c i (6i) 

where F\ , are gaussian fuzzy sets. 

Experimental data sets of N points each are created from (60), with a 2 £ [0, 0.20] . This means 
that the noise applied take values between and ±30% of the output nominal value, which 
is an acceptable practical percentage of noise. These data sets are presented to the proposed 
algorithm, for obtaining an IV fuzzy model, and to the LS based algorithm, for obtaining a LS 
fuzzy model. The models are obtained by the global and local approaches as in (45) and (46), 
repectively. The noise influence is analized according to the difference between the outputs 
of the fuzzy models, obtained from the noisy experimental data, and the output of the plant 
without noise. The antecedent parameters and the structure of the fuzzy models are the same 
in the experiments, while the consequent parameters are obtained by the proposed method 
and by the LS method. Thus, the obtained results are due to these algorithms and accuracy 
conclusions will be derived about the proposed algorithm performance in the presence of 
noise. Two criteria, widely used in experimental data analysis, are applied to avaliate the 
obtained fuzzy models fit: Variance Accounted For (VAF) 



VAF(%) = 100 x 



var{\-Y) 

var{\) 



(62) 



where Y is the nominal plant output, Y is the fuzzy model output and var means signal 
variance, and Mean Square Error (MSE) 



1 N 

MSE = M !>>c - ^t) 2 ( 63 ) 

k=l 
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where y^ is the nominal plant output, yj- is the fuzzy model output and N is the number of 
points. Once obtained these values, a comparative analysis will be established between the 
proposed algorithm, based on IV, and the algorithm based on LS according to the approaches 
presented above. In the performance of the TS models obtained off-line according to (45) 
and (46), the number of points is 500, the proposed algorithm used A equal to 0.99; the 
number of rules is 4, the structure is the presented in (61) and the antecedent parameters 
are obtained by the ECM method for both algorithms. The proposed algorithm performs 
better than the LS algorithm for the two approaches as it is more robust to noise. This 
is due to the chosen instrumental variable matrix, with dl = 1, to satisfy the convergence 
conditions as well as possible. In the global approach, for low noise variance, both algorithms 
presented similar performance with VAF and MSE of 99.50% and 0.0071 for the proposed 
algorithm and of 99.56% and 0.0027 for the LS based algorithm, respectively. However, when 
the noise variance increases, the chosen instrumental variable matrix satisfies the convergence 
conditions, and as a consequence the proposed algorithm becomes more robust to the noise 
with VAF and MSE of 98.81% and 0.0375. On the other hand the LS based algorithm presented 
VAF and MSE of 82.61% and 0.4847, respectively, that represents a poor performance. Similar 
analysis can be applied to the local approach: increasing the noise variance, both algorithms 
present good performances where the VAF and MSE values increase too. This is due to the 
polytope property, where the obtained models can represent local approximations giving 
more flexibility curves fitting. The proposed algorithm presented VAF and MSE values of 
93.70% and 0.1701 for the worst case and of 96.3% and 0.0962 for the better case. The LS based 
algorithm presented VAF and MSE values of 92.4% and 0.2042 for the worst case and of 95.5% 
and 0.1157 for the better case. The worst case of noisy data set was still used by the algorithm 
proposed in (Wang & Langari, 1995), where the VAF and MSE values were of 92.6452% and 
0.1913, and by the algorithm proposed in (Pedrycz, 2006) where the VAF and MSE values were 
of 92.5216% and 0.1910, respectively. These results, considering the local approach, show that 
they have an intermediate performance between the proposed method in this paper and the 
LS based algorithm. For the global approach, the VAF and MSE values are 96.5% and 0.09 
for the proposed method and of 81.4% and 0.52 for the LS based algorithm, respectively. For 
the local approach, the VAF and MSE values are 96.0% and 0.109 for the proposed method 
and of 95.5% and 0.1187 for the LS based algorithm, respectively. In sense to be clear to the 
reader, the results of local and global estimation to the TS fuzzy model from the stochastic 
SISO nonlinear system identification, it has the following conclusions: When interpreting TS 
fuzzy models obtained from data, one has to be aware of the tradeoffs between local and 
global estimation. The TS fuzzy models estimated by local approach describe properly the 
local behavior of the nonlinear system, but not give a good fit; for the global approach, the 
opposite holds - a perfect fit is obtained, but the TS fuzzy models are not relevant for the local 
behavior of the nonlinear system. This is the tradeoffs between local and global estimation. 
To illustrate the robustness of the FIV algorithm, it was performed a numerical experiment 
based on 300 different realizations of noise. The numerical experiment followed a particular 
computational pattern: 

• Define a domain with 300 different sequences of noise; 

• Generate a realization of noise randomly from the domain, and perform the identification 
procedure for the IV and LS based algorithms; 
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Aggregate the results of IV and LS algorithms according to VAF and MSE criteria into the 
final result from histograms, indicating the number of its occurences (frequency) during 
the numerical experiment. 
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Fig. 3. Robustness analysis: Histogram of VAF for the IV and LS based algorithms. 

The IV and LS based algorithms were submitted to these different conditions of noise at 
same time and the efficiency was observed through VAF and MSE criteria according to the 
histograms shown on Fig. 3 and Fig. 4, respectively. Clearly, the proposed method presented 
the best results compared with LS based algorithm. For the global approach, the results of 
VAF and MSE values are of 98.60 ± 1.25% and 0.037 ± 0.02 for the proposed method and of 
84.70 ± 0.65% and 0.38 ± 0.15 for the LS based algorithm, respectively. For the local approach, 
the results of VAF and MSE values are of 96.70 ± 0.55% and 0.07 ± 0.015 for the proposed 
method and of 95.30 ± 0.15% and 0.1150 ± 0.005 for the LS based algorithm, respectively. 
In general, from the results shown in Tab. 1, it can conclude that the proposed method 
has favorable results compared with existing techniques and good robustness properties for 
identification of stochastic nonlinear systems. 



5.2 Experimental results 

In this section, the experimental results on adaptive model based control of a multivariable 
(two inputs and one output) nonlinear pH process, commonly found in industrial 
environment, are presented. 
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Fig. 4. Robustness analysis: Histogram of MSE for the IV and LS based algorithms. 

5.2.1 Fuzzy adaptive black box fuzzy model based control of pH neutralization process 

The input-output experimental data set of the nonlinear plant were obtained from DAISY 1 
(Data Acquisition For Identification of Systems) plataform. 

This plant presents the following input-output variables: 

• Ui(t): acid flow (/); 

• Ui(t): base flow (I); 

• y{t): level of pH in the tank. 

Figure 5 shows the open loop temporal response of the plant, considering a sampling time of 
10 seconds. These data will be used for modeling of the process. The obtained fuzzy model 
will be used for indirect multivariable adaptive fuzzy control design. The TS fuzzy inference 
system uses a functional expression of the pH level in the tank. The i |' =1 < 2 <---''-th rule of the 
multivariable TS fuzzy model, where / is the number of rules is given by: 



R' : IF Y(z)z- 1 is FL, , . THEN 



(64) 



1 accessed in http://homes.esat.kuleuven.be/ smc/daisy. 
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Fig. 5. Open loop temporal response of the nonlinear pH process 

The C-means fuzzy clustering algorithm was used to estimate the antecedent parameters 
of the TS fuzzy model. The fuzzy recursive instrumental variable algorithm based on QR 
factorization, was used to estimate the consequent submodels parameters of the TS fuzzy 
model. For initial estimation was used 100 points, the number of rules was / = 2, and the 
fuzzy frequency response validation method was used for fuzzy controller design based on 
the inverse model (Serra & Ferreira, 2011). 

The parameters of the submodels in the consequent proposition of the multivariable TS 
fuzzy model are shown in Figure 6. It is observed that in addition to nonlinearity, the pH 
neutralization process presents uncertainty behavior in order to commit any application of fix 
control design. 
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Fig. 6. TS fuzzy model parameters estimated by fuzzy instrumental variable algortihm based 
on QR factoration 
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The TS multivariable fuzzy model, at last sample, is given by: 

R 1 :IFy(fc-l)isF 1 THEN 
y l {k) = 1.1707y(k - 1) - 0.2187y(A: - 2) + 0.0372^ (fc) + 0.1562u 2 (k) 

R 2 :IFy(fc-l)isF 2 THEN 
y 2 {k) = 1.0919y(fc - 1) - 0.1861y(fc - 2) + O.CBfMtt^fc) + 0A663u 2 {k) 



(65) 



The validation of the TS fuzzy model, according to equation (65) via fuzzy frequency response 




ff m* f*tyM< 



Fig. 7. Recursive estimation processing for submodels parameters in the TS multivariable 
fuzzy model consequent proposition. 

is shown in Figure 8. It can be observed the efficiency of the proposed identification algorithm 
to track the output variable of pH neutralization process. This result has fundamental 
importance for multivariable adaptive fuzzy controller design step. The region of uncertainty 
defined by fuzzy frequency response for the identified model contains the frequency response 
of the pH process. It means that the fuzzy model represents the dynamic behavior 
perfectly, considering the uncertainties and nonlinearities of the pH neutralization process. 
Consequently, the model based control design presents robust stability characteristic. The 
adaptive control design methodology adopted in this paper consists of a control action based 
on the inverse model. Once the plant model becomes known precisely by the rules of 
multivariable TS fuzzy model, considering the fact that the submodels are stable, one can 
develop a strategy to control the flow of acid and base, in order to maintain the pH level of 
7. Thus, the multivariable fuzzy controller is designed so that the control system closed-loop 
presents unity gain and the output is equal to the reference. So, it yields: 



Gmf(z) 



R(z) _ G' Cl G pi + G[ 2 G l n 
Y(z) \ + Gi 1 G i Pl + Gi 1 G i Pl 



(66) 



where G' c e G£, are the transfer functions of the controllers in the z'-th rule, as G'„ and G'„ 
are submodels in the consequent proposition from the output Y(z) to inputs U\(z) and U2{z), 
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Fig. 8. Validation step of the multivariable TS fuzzy model: (a) - (b) Fuzzy frequency 
response of the TS fuzzy model (black curve) representing the dynamic behavior of the pH 
level and the flow of acid solution (red curve), (c) - (d) Fuzzy frequency response of the TS 
fuzzy model (black curve) representing the dynamic behavior of the pH level and flow of the 
base (red curve). 

respectively. Considering 



and 



results: 



this is, 



Gmf{z) 



y(z) 
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Y(z) - 
lR(z) 



(67) 



(68) 



For compensation this closed loop gain of the control system, it is necessary generate a 
reference signal so that V(z) = R(z). Therefore, adopting the new reference signal -R'(z) = 
§R(z), it yields: 



Y(z) 
Y(z) = 



3 

23 

32 

Y(z) = R(z) 






(69) 

(70) 
(71) 



For the inverse model based indirect multivariable fuzzy control design, one adopte a new 
reference signal given by R'(z) = |R(z). The TS fuzzy multivariable controller presents the 



Highlighted Aspects from Black Box Fuzzy Modeling for Advanced Control Systems Design 

folowing structure: 

-1 ;„ d 



25 



R' : IF Y(z)z _1 is F!,«, > , THEN 



G> 



1 — a\z l — akz 2 



-E(z) 



1 — a\ z 1 — akz 2 , , 
G' = *-*, 2 - E z 



(72) 



The temporal response of the TS fuzzy multivariable adaptive control is shown in Fig. 9. It 
can be observed the control system track the reference signal, pH = 7, because the controller 
can tune itself based on the identified TS fuzzy multivariable model. 



I 



-*-ta 



0.5 



1.5 2 2.5 3 3.5 4 

Time (hours) 



Fig. 9. Performance of the TS fuzzy multivariable adaptive control system. 
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1. Introduction 

Distributed networks have received much attention in the last year because of their 
flexibility and computational performance. The ability to coordinate agents is important in 
many real-world tasks where it is necessary for agents to exchange information with each 
other. Synchronization behavior among agents is found in flocking of birds, schooling of 
fish, and other natural systems. Work has been done to develop cooperative control 
methods for consensus and synchronization (Fax and Murray, 2004; Jadbabaie, Lin and 
Morse, 2003; Olfati-Saber, and Murray, 2004; Qu, 2009; Ren, Beard, and Atkins, 2005; Ren, 
and beard, 2005; Ren, and Beard, 2008; Tsitsiklis, 1984). See (Olfati-Saber, Fax, and Murray, 
2007; Ren, Beard, and Atkins, 2005) for surveys. Leaderless consensus results in all nodes 
converging to common value that cannot generally be controlled. We call this the 
cooperative regulator problem. On the other hand the problem of cooperative tracking 
requires that all nodes synchronize to a leader or control node (Hong, Hu, and Gao, 2006; Li, 
Wang, and Chen, 2004; Ren, Moore, and Chen, 2007; Wang, and Chen, 2002). This has been 
called pinning control or control with a virtual leader. Consensus has been studied for 
systems on communication graphs with fixed or varying topologies and communication 
delays. 

Game theory provides an ideal environment in which to study multi-player decision and 
control problems, and offers a wide range of challenging and engaging problems. Game 
theory (Tijs, 2003) has been successful in modeling strategic behavior, where the outcome 
for each player depends on the actions of himself and all the other players. Every player 
chooses a control to minimize independently from the others his own performance 
objective. Multi player cooperative games rely on solving coupled Hamilton-Jacobi (HJ) 
equations, which in the linear quadratic case reduce to the coupled algebraic Riccati 
equations (Basar, and Olsder, 1999; Freiling, Jank, and Abou-Kandil, 2002; Gajic, and Li, 
1988). Solution methods are generally offline and generate fixed control policies that are 
then implemented in online controllers in real time. These coupled equations are difficult to 
solve. 
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Reinforcement learning (RL) is a sub-area of machine learning concerned with how to 
methodically modify the actions of an agent (player) based on observed responses from its 
environment (Sutton, and Barto, 1998). RL methods have allowed control systems 
researchers to develop algorithms to learn online in real time the solutions to optimal 
control problems for dynamic systems that are described by difference or ordinary 
differential equations. These involve a computational intelligence technique known as 
Policy Iteration (PI) (Bertsekas, and Tsitsiklis, 1996), which refers to a class of algorithms 
with two steps, policy evaluation and policy improvement. PI has primarily been developed for 
discrete-time systems, and online implementation for control systems has been developed 
through approximation of the value function (Bertsekas, and Tsitsiklis, 1996; Werbos, 1974; 
Werbos, 1992). PI provides effective means of learning solutions to HJ equations online. In 
control theoretic terms, the PI algorithm amounts to learning the solution to a nonlinear 
Lyapunov equation, and then updating the policy through minimizing a Hamiltonian 
function. Policy Iteration techniques have been developed for continuous-time systems in 
(Vrabie, Pastravanu, Lewis, and Abu-Khalaf, 2009). 

RL methods have been used to solve multiplayer games for finite-state systems in (Busoniu, 
Babuska, and De Schutter, 2008; Liftman, 2001). RL methods have been applied to learn 
online in real-time the solutions for optimal control problems for dynamic systems and 
differential games in (Dierks, and Jagannathan, 2010; Johnson, Hiramatsu, Fitz-Coy, and 
Dixon, 2010; Vamvoudakis 2010; Vamvoudakis 2011). 

This book chapter brings together cooperative control, reinforcement learning, and game 
theory to solve multi-player differential games on communication graph topologies. There 
are four main contributions in this chapter. The first involves the formulation of a graphical 
game for dynamical systems networked by a communication graph. The dynamics and value 
function of each node depend only on the actions of that node and its neighbors. This 
graphical game allows for synchronization as well as Nash equilibrium solutions among 
neighbors. It is shown that standard definitions for Nash equilibrium are not sufficient for 
graphical games and a new definition of "Interactive Nash Equilibrium" is given. The 
second contribution is the derivation of coupled Riccati equations for solution of graphical 
games. The third contribution is a Policy Iteration algorithm for solution of graphical games 
that relies only on local information from neighbor nodes. It is shown that this algorithm 
converges to the best response policy of a node if its neighbors have fixed policies, and to 
the Nash solution if all nodes update their policies. The last contribution is the development 
of an online adaptive learning algorithm for computing the Nash equilibrium solutions of 
graphical games. 

The book chapter is organized as follows. Section 2 reviews synchronization in graphs and 
derives an error dynamics for each node that is influenced by its own actions and those of its 
neighbors. Section 3 introduces differential graphical games cooperative Nash equilibrium. 
Coupled Riccati equations are developed and stability and solution for Nash equilibrium are 
proven. Section 4 proposes a policy iteration algorithm for the solution of graphical games 
and gives proofs of convergence. Section 5 presents an online adaptive learning solution 
based on the structure of the policy iteration algorithm of Section 4. Finally Section 6 
presents a simulation example that shows the effectiveness of the proposed algorithms in 
learning in real-time the solutions of graphical games. 
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2. Synchronization and node error dynamics 

2.1 Graphs 

Consider a graph G = (V,E) with a nonempty finite set of N nodes V = {v 1 ,---,v N } and a set 
of edges or arcs EcVxV . We assume the graph is simple, e.g. no repeated edges and 
(Pj,t> ( -) ^£,Vt no self loops. Denote the connectivity matrix as E = [e ; :] with 

e, > if {Vi,Vi) e E and e ; = otherwise. Note e„ = . The set of neighbors of a node v { is 
N, ={Vj :(5j,D;)eE) , i.e. the set of nodes with arcs incoming to v t . Define the in-degree 
matrix as a diagonal matrix D = diag(d t ) with d t = V e t : the weighted in-degree of node i 

(i.e. i -th row sum of E). Define the graph Laplacian matrix as L =D~E , which has all row 
sums equal to zero. 

A directed path is a sequence of nodes v 0/ v 1 ,---,v r such that (v t ,v i+1 ) eE,i e{0,l,---,r -1} . 

A directed graph is strongly connected if there is a directed path from u, to c for all 

distinct nodes V t ,Vj eV . A (directed) tree is a connected digraph where every node except 

one, called the root, has in-degree equal to one. A graph is said to have a spanning tree if a 
subset of the edges forms a directed tree. A strongly connected digraph contains a spanning 
tree. 

General directed graphs with fixed topology are considered in this chapter. 

2.2 Synchronization and node error dynamics 

Consider the N systems or agents distributed on communication graph G with node 
dynamics 

X { = Ax t + B i u l (1) 

where x,(f)eK" is the state of node i, H > (t)eR m ' its control input. Cooperative team 
objectives may be prescribed in terms of the local neighborhood tracking error S { e R" (Khoo, 
Xie, and Man, 2009) as 

#i = X e 'j ( x > " *;') + S, (*,■ - x o ) ( 2 ) 

jsN, 

The pinning gain g t > is nonzero for a small number of nodes i that are coupled directly to 
the leader or control node x , and g i > for at least one i (Li, Wang, and Chen, 2004). We 
refer to the nodes i for which g i ^0 as the pinned or controlled nodes. Note that 8 i 
represents the information available to node i for state feedback purposes as dictated by the 
graph structure. 

The state of the control or target node is x (t) e R" which satisfies the dynamics 
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v o 



-.Ax (3) 



Note that this is in fact a command generator (Lewis, 1992) and we seek to design a 
cooperative control command generator tracker. Note that the trajectory generator A may 
not be stable. 

The Synchronization control design problem is to design local control protocols for all the 
nodes in G to synchronize to the state of the control node, i.e. one requires x,(f) — > x (t), V/ . 

From (2), the overall error vector for network Gr is given by 

S = ((L + G)®I n )(x-x D ) = ((L + G)®I n )£ (4) 

where the global vectors are 

* = [*[ x[ •■■ 4f 6l ° N s = \_ s l Sl ■■■ 4] r e»" N and x = Lr eK" N , with 
I_ = 10I n eR" Nx " and 1 the N-vector of ones. The Kronecker product is <8> (Brewer, 1978). 
G gR x is a diagonal matrix with diagonal entries equal to the pinning gains g i . The 

(global) consensus or synchronization error (e.g. the disagreement vector in (Olfati-Saber, 

and Murray, 2004)) is 

t=(x-x ) e R" N (5) 

The communication digraph is assumed to be strongly connected. Then, if g i =£ for at least 

one i , (L + G) is nonsingular with all eigenvalues having positive real parts (Khoo, Xie, and 

Man, 2009). The next result therefore follows from (4) and the Cauchy Schwartz inequality 
and the properties of the Kronecker product (Brewer, 1978). 

Lemma 1. Let the graph be strongly connected and G ^ . Then the synchronization error is 
bounded by 

\\C\\<\\S\\/*(L + G) (6) 

with a(L + G) the minimum singular value of (L + G) , and S(t) = if and only if the nodes 
synchronize, that is 

x(t) = I_x (t) (7) 



Our objective now shall be to make small the local neighborhood tracking errors S t (t) , which 
in view of Lemma 1 will guarantee synchronization. 

To find the dynamics of the local neighborhood tracking error, write 

S,=AS 1 + (d 1+ g I )B I u 1 -^e 11 B ] u J (8) 
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with <J ! eK",M / eK ra ',Vi. 

This is a dynamical system with multiple control inputs, from node i and all of its neighbors. 

3. Cooperative multi-player games on graphs 

We wish to achieve synchronization while simultaneously optimizing some performance 
specifications on the agents. To capture this, we intend to use the machinery of multi-player 
games (Basar, Olsder, 1999). Define u G _, = {u, : j e N,j * ;} as the set of policies of all other 

nodes in the graph other than node i. Define u_ t (t) as the vector of the control inputs 
{u, : j e N,} of the neighbors of node i. 

3.1 Cooperative performance index 

Define the local performance indices 

J i {S i {0),u i ,u_ i ) = ±l(S[Q ii S i +ujR ii u i + X «[Vy)* -iJM$(0,«*(0,M0)* ( 9 ) 

o ;'eNj o 

where all weighting matrices are constant and symmetric with Q u > 0,R U > 0,R, > . Note 

that the j-th performance index includes only information about the inputs of node i and its 
neighbors. 

For dynamics (8) with performance objectives (9), introduce the associated Hamiltonians 

f \ 

HifaPiW^P? AS i +(d i+gi )B i u i -^e ij B j u j +\sjQ ii 6 i +{ujR il u i +^ujR ij u j =0 (10) 



jeN, 



jeN i 



where p ; is the costate variable. Necessary conditions (Lewis, and Syrmos, 1995) for a 
minimum of (9) are (1) and 

-i> t =^-'A T Pi+QuS, (11) 

= ^^u 1 =-(d 1+gl )R^B, T p 1 (12) 

du t 

3.2 Graphical games 

Interpreting the control inputs m,,m as state dependent policies or strategies, the value 
function for node i corresponding to those policies is 

00 

ViW)) = \\{SjQA + ujRaUi + E "JV;) dt < 13 ) 
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Definition 1. Control policies u t , Vz are defined as admissible if u i are continuous, 
m,(0) = , u i stabilize systems (8) locally, and values (13) are finite. 

When Vj is finite, using Leibniz' formula, a differential equivalent to (13) is given in terms 
of the Hamiltonian function by the Bellman equation 



8V- dV T( 

' 8& ' ' dS: 



AS,+(d, +g,)B i u i - X e lj B j u j +\SjQ^ t + K^", + f 1 «[tyy = ° < 14 ) 



with boundary condition V ; (0) = 0. (The gradient is disabused here as a column vector.) 
That is, solution of equation (14) serves as an alternative to evaluating the infinite integral 
(13) for finding the value associated to the current feedback policies. It is shown in the Proof 
of Theorem 2 that (14) is a Lyapunov equation. According to (13) and (10) one equates 
Pi=dV { /dS { . 

The local dynamics (8) and performance indices (9) only depend for each node i on its own 
control actions and those of its neighbors. We call this a graphical game. It depends on the 
topology of the communication graph G = (V,E) . We assume throughout the chapter that 
the game is well-formed in the following sense. 



Definition 2. The graphical game with local dynamics (8) and performance indices (9) is 
well-formed if B. ^O^^eE, R, ^ ^ e, e E . 

The control objective of agent i in the graphical game is to determine 



VtW)) = mmfaslQuS, + ujR u u, + £ uJR^) At (15) 

Employing the stationarity condition (12) (Lewis, and Syrmos, 1995) one obtains the control 
policies 

^ = u,(V,) = -(d, + gJBfB? ^±- - -hiipi) (16) 

The game defined in (15) corresponds to Nash equilibrium. 

Definition 3. (Basar, and Olsder, 1999) (Global Nash equilibrium) An N-tuple of policies 
I m 1 ,m 2 ,...,u n > is said to constitute a global Nash equilibrium solution for an N player game 
if for all i s N 

]' i ±] i {u i ,u G _ i )<] i {u i ,u G _ i ) (17) 

The N- tuple of game values 1 7 1 ,/ 2 ,...,/ N } is known as a Nash equilibrium outcome of the N- 
player game. 
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The distributed multiplayer graphical game with local dynamics (8) and local performance 
indices (9) should be contrasted with standard multiplayer games (Abou-Kandil, Freiling, 
Ionescu, and Jank, 2003; Basar, and Olsder 1999) which have centralized dynamics 

N 

z = Az + Y,B,u, (18) 

where z e R" is the state, u ; (f) s K m ' is the control input for every player, and where the 

performance index of each player depends on the control inputs of all other players. In the 
graphical games, by contrast, each node's dynamics and performance index only depends 
on its own state, its control, and the controls of its immediate neighbors. 

It is desired to study the distributed game on a graph defined by (15) with distributed 
dynamics (8). It is not clear in this scenario how global Nash equilibrium is to be achieved. 

Graphical games have been studied in the computational intelligence community (Kakade, 
Kearns, Langford, and Ortitz, 2003; Kearns, Liftman, and Singh, 2001; Shoham, and Leyton- 
Brown, 2009). A (nondynamic) graphical game has been defined there as a tuple (G,U,v) 
with G = (V,E) a graph with N nodes, action set U = U 1 x---xU N with U t the set of actions 

available to node i, and p = [t> 1 •■■ v N ] a payoff vector, with D ( -(U;,{IL :j eNJ) eR the 

payoff function of node i. It is important to note that the payoff of node i only depends on its own 
action and those of its immediate neighbors. The work on graphical games has focused on 
developing algorithms to find standard Nash equilibria for payoffs generally given in terms 
of matrices. Such algorithms are simplified in that they only have complexity on the order of 
the maximum node degree in the graph, not on the order of the number of players N. 
Undirected graphs are studied, and it is assumed that the graph is connected. 

The intention in this chapter is to provide online real-time adaptive methods for solving 
differential graphical games that are distributed in nature. That is, the control protocols and 
adaptive algorithms of each node are allowed to depend only information about itself and 
its neighbors. Moreover, as the game solution is being learned, all node dynamics are 
required to be stable, until finally all the nodes synchronize to the state of the control node. 
These online methods are discussed in Section V. 

The following notions are needed in the study of differential graphical games. 

Definition 4. (Shoham, and Leyton-Brown, 2009) Agent i's best response to fixed policies u_ t 
of his neighbors is the policy u i such that 



} i {u il u- i )<} i {u il u_ i ) (19) 



for all policies u t of agent i. 



For centralized multi-agent games, where the dynamics is given by (18) and the 
performance of each agent depends on the actions of all other agents, an equivalent 
definition of Nash equilibrium is that each agent is in best response to all other agents. In 
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graphical games, if all agents are in best response to their neighbors, then all agents are in 
Nash equilibrium, as seen in the proof of Theorem 1. 

However, a counterexample shows the problems with the definition of Nash equilibrium 
in graphical games. Consider the completely disconnected graph with empty edge set 
where each node has no neighbors. Then Definition 4 holds if each agent simply chooses 

his single-player optimal control solution 7, =/,( M ! ), since, for the disconnected graph 
case one has 

Ji ("i ) = /, ( u , > u g-i) = Ji ( u ,> u 'g-,)> v * (20) 

for any choices of the two sets Mg-i' m 'g-! °f the policies of all the other nodes. That is, the 
value function of each node does not depend on the policies of any other nodes. 

Note, however, that Definition 3 also holds, that is, the nodes are in a global Nash 
equilibrium. Pathological cases such as this counterexample cannot occur in the standard 
games with centralized dynamics (18), particularly because stabilizability conditions are 
usually assumed. 

3.3 Interactive Nash equilibrium 

The counterexample in the previous section shows that in pathological cases when the 
graph is disconnected, agents can be in Nash equilibrium, yet have no influence on each 
others' games. In such situations, the definition of coalition-proof Nash equilibrium 
(Shinohara, 2010) may also hold, that is, no set of agents has an incentive to break away 
from the Nash equilibrium and seek a new Nash solution among themselves. 

To rule out such undesirable situations and guarantee that all agents in a graph are involved 
in the same game, we make the following stronger definition of global Nash equilibrium. 



Definition 5. (Interactive Global Nash equilibrium) An N-tuple of policies lu 1 ,u 2 ,...,w N } is 

said to constitute an interactive global Nash equilibrium solution for an N player game if, 
for all ieN , the Nash condition (17) holds and in addition there exists a policy u\ such 
that 

Ji(u'k> u G-k)*Ji( u \> u G-k) (21) 

for all i,k eN . That is, at equilibrium there exists a policy of every player k that influences 
the performance of all other players i. 

If the systems are in Interactive Nash equilibrium, the graphical game is well-defined in the 
sense that all players are in a single Nash equilibrium with each player affecting the 
decisions of all other players. Condition (21) means that the reaction curve (Basar, and 
Olsder, 1999) of any player i is not constant with respect to all variations in the policy of any 
other player k. 
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The next results give conditions under which the local best responses in Definition 4 imply 
the interactive global Nash of Definition 5. 

Consider the systems (8) in closed-loop with admissible feedbacks (12), (16) denoted by 
u k = K k p k - v k for a single node k and u j = KjPj' V/ * k . Then 

4 = ASi + (d, + g 1 )B 1 K lPl - £ e ij B j K j p } + e ik B k v k , k * i (2 2) 

jsN, 

The global closed-loop dynamics are 



(In 9 A) ((L + G)9I„)diag(B i K l ) 



-diag(Qu) 



-(In® A 1 ) 



((L + G)®I n )B k 




v k 'A 



1v k (23) 



with B k = diag(B t ) and v k = •■■ v k •■■ has all block entries zero with v k in block 

k. Consider node i and let M > be the first integer such that [(L + G) ] ft ^ , where [,] ft 
denotes the element (i,k) of a matrix. That is, M is the length of the shortest directed path 
from k to i. Denote the nodes along this path by k = ^0'^l''"'^M-l'^M = ' ■ Denote element 
(i,k) of L + G by i jk . Then the nxm block element in block row i and block column k of 
matrix A 2 ^' 1 ^ is equal to 



[A m ' 1) B]*= I i^-t^B^K^Q 



M-l ^ K M-1 K M-2 



K k O k B k 



k t k^kj k Zj *m-i ^m-i^ 



(24) 



where B^ fc s 
matrix. 
Assumption 1. 



a. B, e _R m '«-i xm * has rank m. 



and [ ] denotes the position of the block element in the block 



All shortest paths to node i from node k pass through a single neighbor k M - 1 of i. 

An example case where Assumption la holds is when there is a single shortest path from k 
to i, m, = m, Vi , rank(B t ) = m, Vi . 

Lemma 2. Let (A,B.) be reachable for all j eN and let Assumption 1 hold. Then the i-th 
closed-loop system (22) is reachable from input v k if and only if there exists a directed path 
from node k to node i. 

Proof: 

Sufficiency. If k = i the result is obvious. Otherwise, the reachability matrix from node k to 
node i has the nxm block element in block row i and block column k given as 
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|"^2(M-l)g ^2(M-l)+lg ^2(M-l)+2g l 3 

B, 



YB k Y AB k Y A 2 B k 

«M-1 *M-1 *M-1 



B 



k M _i ,k 

o \ k 



where * denotes nonzero entries. Under the assumptions, the matrix on the right has full 
row rank and the matrix on the left is written as fg, AB t A l B h ■ ■ -1 ■ 

However, M g \ is reachable. 

Necessity. If there is no path from node k to node i, then the control input of node k cannot 
influence the state or value of node i. 



Theorem 1. Let (A,Bj) be reachable for all i sN . Let every node i be in best response to all 
his neighbors j e N, . Let Assumption 1 hold. Then all nodes in the graph are in interactive 
global Nash equilibrium if and only if the graph is strongly connected. 

Proof: 

Let every node i be in best response to all his neighbors j e N, . Then 

/, (u*,a H ) < /, («,,«_,), Vi ■ Hence u . = u * \/ u e u _. and /, («*,«!,) < /, («,■,«!,•), Vz . However, 

according to (9) /. (u*,^^^ = J { (w*,^,^), Vic g |i]wN, so that /. (m*,«g_,) < /, (m,-,«g-,)' v ' 
and the nodes are in Nash equilibrium. 

Necessity. If the graph is not strongly connected, then there exist nodes k and i such that 
there is no path from node k to node i. Then, the control input of node k cannot influence the 
state or the value of node i. Therefore, the Nash equilibrium is not interactive. 

Sufficiency. Let (A,B t ) be reachable for all i eN . Then if there is a path from node k to node 
i, the state 8 { is reachable from u k , and from (9) input u k can change the value /, . Strong 
connectivity means there is a path from every node k to every node i and condition (21) 
holds for all i,k e N . 



The reachability condition is sufficient but not necessary for Interactive Nash equilibrium. 
According to the results just established, the following assumptions are made. 
Assumptions 2. 

a. (A,B t ) is reachable for all ieN . 

b. The graph is strongly connected and at least one pinning gain g i is nonzero. Then 
(L + G) is nonsingular. 
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3.4 Stability and solution of graphical games 

Substituting control policies (16) into (14) yields the coupled cooperative game Hamilton- 
Jacobi (HJ) equations 

^A* +UjQ ii 5 i +Ud, +gi f^L B^B^+\ V (d, +g,) 2 ^ BJG 1 2UE 1 B < r ^- = 0,«eN (25) 



where the closed-loop matrix is 



■dV, .. v *,a _, .. « „-i„ r 0V ) 



A^A^-id^gfB^B^ *Ze lj (d }+ g j )B j Rp j T -J-,ieN (26) 

For a given V, , define m, = Mj(V)) as (16) given in terms of V) . Then HJ equations (25) can 
be written as 

H i (S i ,jj-,ulu_ i ) = (27) 

There is one coupled HJ equation corresponding to each node, so solution of this N-player 
game problem is blocked by requiring a solution to N coupled partial differential equations. 
In the next sections we show how to solve this N-player cooperative game online in a 
distributed fashion at each node, requiring only measurements from neighbor nodes, by 
using techniques from reinforcement learning. 

It is now shown that the coupled HJ equations (25) can be written as coupled Riccati 
equations. For the global state 8 given in (4) we can write the dynamics as 

8 = (I N <S> A)S + (L + G) ® I„*flg(B; )u (28) 

where u is the control given by 

u = -diag(R i r l Bj){(p + G)®I nV ) (29) 

where diag(.) denotes diagonal matrix of appropriate dimensions. Furthermore the global 
costate dynamics are 

dH T 

-P = -^^(lN®Ayp + diag(Q ii )8 (30) 

This is a set of coupled dynamic equations reminiscent of standard multi-player games 
(Basar, and Olsder, 1999) or single agent optimal control (Lewis, and Syrmos, 1995). 
Therefore the solution can be written without any loss of generality as 

p = P8 (31) 
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for some matrix P > s R" Nx " N . 

Lemma 3. HJ equations (25) are equivalent to the coupled Riccati equations 

S T P T A,S - S T P T B,PS + \S T Q,S + ±S T P T R l PS = 
or equivalently, in closed-loop form, 



(32) 



(P 1 A ic + A lc l P + Q,+ P l R,P) = 



(33) 



where P is defined by (31), and 



[A]" 







diag^ + g^R^B, 7 ) 



A i c = A i ~B i P 



Qi = 



[Qsf 







.Ri^diagQidi+giWR?) 



R,i 



diag((d,+g,)R?B, T ) 



Proof: 

Take (14) and write it with respect to the global state and costate as 

"0 



H, 



5V, 



BS 1 



3Vn 
8S A , 







8V 1 



8Vn 
dS N 



•■■ 

; o 







(34) 
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.1*7 



\Quf 



S + \ 



By definition of the costate one has 



8V 1 
dd x 



dS N 



PS 



(35) 



From the control policies (16), (34) becomes (32). 

It is now shown that if solutions can be found for the coupled design equations (25), they 
provide the solution to the graphical game problem. 

Theorem 2. Stability and Solution for Cooperative Nash Equilibrium. 

Let Assumptions 1 and 2a hold. Let Vj > e C , i e N be smooth solutions to HJ equations 
(25) and control policies u { , i e N be given by (16) in terms of these solutions Vj . Then 

a. Systems (8) are asymptotically stable so all agents synchronize. 

m 1 ,m 2 ,...,m n [ are in global Nash equilibrium and the corresponding game values are 



7<($(0))=V,,» g N 



(36) 



Proof: 



If Vj > satisfies (25) then it also satisfies (14). Take the time derivative to obtain 



8V: T ■ dV i Tf 



V i = TF % = TF\ A5 ' + W + ^ B ' Ui ~ £ e v B i u i 



II xT. 

2 



S[Q,,S, +u]R li u l +Y j u T j R lj u j \ (37) 



which is negative definite since Q„ > . Therefore Vj is a Lyapunov function for <5 ; and 
systems (8) are asymptotically stable. 

According to part a, S t (t)— >0 for the selected control policies. For any smooth functions 
Vj($), i e N , such that Vj(0) = , setting Vj(<5j(°o)) = one can write (9) as 



/,($(<>),«„«_,) = A J (^, T Q n A + », r R, Mi + X «[V>) df + V < W°» 



°°r5V ^ 

^I jeNj 
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Now let V t satisfy (25) and u t ,w_ ( - be the optimal controls given by (16). By completing the 
squares one has 

/ J (j 4 (o),« i ,«- J )=v < w(o))+J(| Z («/-«/) Tr #(«/-«/) + 2(«*-"«*) Tr «( m «-«<*) 

7'eN; 
-^ Z e »; B ;(«/-"/) + Z "7 R »;("; "«/))<*' 
At the equilibrium point m,- = m ; and u ■ = m - so 

/,*($(<>),«/,«_/) =V,($(0)) 
Define 

00 

/,(«,,«_,*) =V, (^(0)) + !/(«,- -Ui'fMu, -u-)dt 

o 

and /, = V, (^(0)) . Then clearly /, and /;(«,•, M_,) satisfy (19). Since this is true for all t, Nash 
condition (17) is satisfied. 

■ 

The next result shows when the systems are in Interactive Nash equilibrium. This means 
that the graphical game is well defined in the sense that all players are in a single Nash 
equilibrium with each player affecting the decisions of all other players. 

Corollary 1. Let the hypotheses of Theorem 2 hold. Let Assumptions 1 and 2 hold so that the 

graph is strongly connected. Then |m 1 ,m 2 ,...,m n | are in interactive Nash equilibrium and all 

agents synchronize. 

Proof: 

From Theorems 1 and 2. 



3.5 Global and local performance objectives: Cooperation and competition 

The overall objective of all the nodes is to ensure synchronization of all the states x { {t) to 

x (t) . The multi player game formulation allows for considerable freedom of each agent 

while achieving this objective. Each agent has a performance objective that can embody 
team objectives as well as individual node objectives. 

The performance objective of each node can be written as 



•' ~ n 2-i h + n 2-i Ui >j' ~ ' team + '* 



conflict 
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where ] team is the overall ('center of gravity') performance objective of the networked team 
and /; 1L is the conflict of interest or competitive objective. ] team measures how much the 

players are vested in common goals, and y° n ' lc expresses to what extent their objectives 

differ. The objective functions can be chosen by the individual players, or they may be 
assigned to yield some desired team behavior. 

4. Policy iteration algorithms for cooperative multi-player games 

Reinforcement learning (RL) techniques have been used to solve the single-player optimal 
control problem online using adaptive learning techniques to determine the optimal value 
function. Especially effective are the approximate dynamic programming (ADP) methods 
(Werbos, 1974; Werbos, 1992). RL techniques have also been applied for multiplayer games 
with centralized dynamics (18). See for example (Busoniu, Babuska, and De Schutter, 2008; 
Vrancx, Verbeeck, and Nowe, 2008). Most applications of RL for solving optimal control 
problems or games online have been to finite-state systems or discrete-time dynamical 
systems. In this section is given a policy iteration algorithm for solving continuous-time 
differential games on graphs. The structure of this algorithm is used in the next section to 
provide online adaptive solutions for graphical games. 

4.1 Best response 

Theorem 2 and Corollary 1 reveal that, under assumptions 1 and 2, the systems are in 
interactive Nash equilibrium if, for all i e N node i selects his best response policy to his 
neighbors policies and the graph is strongly connected. Define the best response HJ 
equation as the Bellman equation (14) with control u { = u { given by (16) and arbitrary 
policies m_, ={uj : j e N,} 

Q = H l {S l , d -^,u l ,u_ l )J-^ A\ + \SjQ n S, +\{d l +g l f 8 -^ B,^B, r 5 + i £ u]R l]U] (38) 
dS, dS t dSi ddi 2 ;eN . 

where the closed-loop matrix is 

A, c = AS, - (d t + gi ?BJ$B? ^ - £ eij B jUl (39) 

Theorem 3. Solution for Best Response Policy 

Given fixed neighbor policies u_ t ={u, : j e N,} , assume there is an admissible policy u i . Let 
V; > e C be a smooth solution to the best response HJ equation (38) and let control policy 
U i be given by (16) in terms of this solution V, . Then 

a. Systems (8) are asymptotically stable so that all agents synchronize. 

b. u, is the best response to the fixed policies u_ { of its neighbors. 
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Proof: 

a. Vj > satisfies (38). Proof follows Theorem 2, part a. 

b. According to part a, S t (t) — » for the selected control policies. For any smooth functions 
Vj(^), i e N , such that V,(0) = , setting V;(£;(oo)) = one can write (9) as 

00 

WMW-i) = \\(SjQA + ujRu* + X "[V;) * + V,($(0)) 

/*, 

°°r5V ^ 

Now let V t satisfy (38), u t be the optimal controls given by (16), and u_ { be arbitrary 
policies. By completing the squares one has 

00 

] i {8 i {Q),u i ,u A ) = V, ($(0)) + Jl(«, - M ,*) T R n ( M , - «;)dt 

o 

The agents are in best response to fixed policies u_ ; when u, = u t so 

/ ( .(^.(0), M *, M .,.) = V ( (^(0)) 
Then clearly /,-(^(0),M,-,M_,-)and /,(^(0), «*,«_;) satisfy (19). 



4.2 Policy iteration solution for graphical games 

The following algorithm for the N-player distributed games is motivated by the structure of 
policy iteration algorithms in reinforcement learning (Bertsekas, and Tsitsiklis, 1996; Sutton, 
and Barto, 1998) which rely on repeated policy evaluation (e.g. solution of (14)) and policy 
improvement (solution of (16)). These two steps are repeated until the policy improvement 
step no longer changes the present policy. If the algorithm converges for every i , then it 
converges to the solution to HJ equations (25), and hence provides the distributed Nash 
equilibrium. One must note that the costs can be evaluated only in the case of admissible 
control policies, admissibility being a condition for the control policy which initializes the 
algorithm. 

Algorithm 1. Policy Iteration (PI) Solution for N-player distributed games. 

Step 0: Start with admissible initial policies u t , Vt . 
Step 1: (Policy Evaluation) Solve for V t using (14) 

H,^,^ ,«,*,«-,*) = 0,V«=1 N (40) 

00: 
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Step 2: (Policy Improvement) Update the N- tuple of control policies using 

uf +1 =argminH,(<y„g- ,«,-,«_,-* ),Vi =1,...,N 
which explicitly is 

u) +1 =-{d l+gl )Kr^ d -^ ,V« = 1 N. (41) 

3d, 

Go to step 1. 

On convergence- End 



The following two theorems prove convergence of the policy iteration algorithm for 
distributed games for two different cases. The two cases considered are the following, i) only 
agent i updates its policy and ii) all the agents update their policies. 

Theorem 4. Convergence of Policy Iteration algorithm when only i"' agent updates its policy 
and all players w_, in its neighborhood do not change. Given fixed neighbors policies m_, , 
assume there exists an admissible policy u i . Assume that agent i performs Algorithm 1 and 
the its neighbors do not update their control policies. Then the algorithm converges to the 
best response m, to policies m_, of the neighbors and to the solution V i to the best response 
HJ equation (38). 

Proof: 

It is clear that 

H°(S i ,^y_ i )^minH i (S i ,^,ulu* i ) = H i (S i ,^,ut + \u k _ i ) (42) 

odj Hi odj odj 

rlV ^ 

Let H,(<y,, — '- ,u* ,ujf) = from (40) then according to (42) it is clear that 
dS { 

H,V„^-,«4)*0 (43) 

Using the next control policy u i + and the current policies m_ ; one has the orbital 



derivative (Leake, Wen Liu, 1967) 



dS: 



From (42) and (43) one has 
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^=l^(S l ,^,u\)-US i ,^\u^-L l {S i ,ur\ui i ) (44) 

3d, 

Because only agent i update its control it is true that u_~l = u_ t and 
But since Vf +1 = -L i (^,M* +1 ,«*t 1 ) , from (44) one has 

v;*=H{ , («y / ,|^*,«^)-L i w,«f +1 ,«! / )^-L < (^,«f +1 ,«! / )=v J * +1 (45) 

So that Vj* < Vf +1 and by integration it follows that 

V t k+1 < V} (46) 

Since V t <V, , the algorithm converges, to V t , to the best response HJ equation (38). 



The next result concerns the case where all nodes update their policies at each step of the 
algorithm. Define the relative control weighting as p f , =a(RJ: -R,.), where (t(R7j R«)is the 

maximum singular value of RJi Ry . 

Theorem 5. Convergence of Policy Iteration algorithm when all agents update their 
policies. Assume all nodes i update their policies at each iteration of PI. Then for small 
enough edge weights e, and p, , m, converges to the global Nash equilibrium and for all 

i , and the values converge to the optimal game values V t — > V i . 

Proof: 

It is clear that 

"iW'/TT- /«i /«-i ) = H i( d i>^- > u -i) + 2L\ u i - u j ) K ij( u i -"; ) 

;sN, vo i jeN; 



and so 



Vi = -Li (Si , u t , m_, ) = -L; (J, , u t , u_, ■) + \ 2_ ( u j - u j ) R ,j ( u j - u j ) 



k+lT 



3v; v- „ , k „ k+i 



°"i jsN, jeN, 
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Therefore, 



^<r4K«/ +1 -»/) lR ,("/ +1 -»/) 

jeN, 



aT7 k+lT 
+ aV i \* ., p I,, ; 1 ,. *\ V <, fcT D /,, k+1 



Z««r-« y »)-I« y %(«r-«y) 



A sufficient condition for V,' c < V,*^ 1 is 



±to j %to j -e ij ( P t +1 ) T B } to r (d j +g j )<p';- 1 )B } T RfR 1i to } >0 



^(R,j)\\^j\\ > e,j \\pi\\ ■ \\Bj\\ + (dj + g } )p n |pH| ■ \\BjW where A Uj = («/ +1 - «/) , p, the costate 
and ff(JRj.-) is the minimum singular value of J?, . 

This holds if e, = 0, /?,■■ = . By continuity, it holds for small values of eu, Pu ■ 



This proof indicates that for the PI algorithm to converge, the neighbors' controls should not 
unduly influence the z'-th node dynamics (8), and the j'-th node should weight its own 
control m in its performance index / relatively more than node i weights u in /, . These 

requirements are consistent with selecting the weighting matrices to obtain proper 
performance in the simulation examples. An alternative condition for convergence in 

Theorem 5 is that the norm B should be small. This is similar to the case of weakly 

coupled dynamics in multi-player games in (Basar, and Olsder, 1999). 

5. Online solution of multi-agent cooperative games using neural networks 

In this section an online algorithm for solving cooperative Hamilton-Jacobi equations (25) 
based on (Vamvoudakis, Lewis 2011) is presented. This algorithm uses the structure in the 
PI Algorithm 1 to develop an actor/ critic adaptive control architecture for approximate 
online solution of (25). Approximate solutions of (40), (41) are obtained using value function 
approximation (VFA). The algorithm uses two approximator structures at each node, which 
are taken here as neural networks (NN) (Abu-Khalaf, and Lewis, 2005; Bertsekas, and 
Tsitsiklis, 1996; Vamvoudakis, Lewis 2010; Werbos, 1974; Werbos, 1992). One critic NN is 
used at each node for value function approximation, and one actor NN at each node to 
approximate the control policy (41). The critic NN seeks to solve Bellman equation (40). We 
give tuning laws for the actor NN and the critic NN such that equations (40) and (41) are 
solved simultaneously online for each node. Then, the solutions to the coupled HJ equations 
(25) are determined. Though these coupled HJ equations are difficult to solve, and may not 
even have analytic solutions, we show how to tune the NN so that the approximate 
solutions are learned online. The next assumption is made. 

Assumption 2. For each admissible control policy the nonlinear Bellman equations (14), (40) 
have smooth solutions V, > . 
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In fact, only local smooth solutions are needed. To solve the Bellman equations (40), 
approximation is required of both the value functions V t and their gradients dV i / d8 i . This 
requires approximation in Sobolev space (Abu-Khalaf, and Lewis, 2005). 

5.1 Critic neural network 

According to the Weierstrass higher-order approximation Theorem (Abou-Khalaf, and 
Lewis, 2005) there are NN weights W t such that the smooth value functions V, are 
approximated using a critic NN as 

V i {S i ) = Wjt i {z i ) + s i (47) 

where z ; (f) is an information vector constructed at node i using locally available 

measurements, e.g. S t (t), {8At):j eN,} . Vectors <zS,(z ; )eIR ! are the critic NN activation 

function vectors, with h the number of neurons in the critic NN hidden layer. According to 
the Weierstrass Theorem, the NN approximation error £, converges to zero uniformly as 

h — > co . Assuming current weight estimates W t , the outputs of the critic NN are given by 

V 1= W^, (48) 

Then, the Bellman equation (40) can be approximated at each step k as 

It is desired to select IV, to minimize the square residual error 

Ei=H, e H, ( 5 °) 

Then YJ i — > W { which solves (49) in a least-squares sense and e H becomes small. Theorem 
6 gives a tuning law for the critic weights that achieves this. 

5.2 Action neural network and online learning 

Define the control policy in the form of an action neural network which computes the 
control input (41) in the structured form 

T 

«, - u 1+N = -1(4 + gl )R?B, T M w i+N (51) 

ds, 

where W i+N denotes the current estimated values of the ideal actor NN weights PV ; . The 
notation U i+N is used to keep indices straight in the proof. Define the critic and actor NN 
estimation errors as W t =W i -W i and W i+N = W t - W i+N . 

The next results show how to tune the critic NN and actor NN in real time at each node so 
that equations (40) and (41) are simultaneously solved, while closed-loop system stability is 
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also guaranteed. Simultaneous solution of (40) and (41) guarantees that the coupled HJ 
equations (25) are solved for each node i. System (8) is said to be uniformly ultimately 
bounded (UUB) if there exists a compact set Scl"so that for all S t (0)eS there exists a 
bound B and a time T(B,5j(0)) such that |<?,(f)||<B for all t>t +T. 

Select the tuning law for the i th critic NN as 

^ = -"' S" = -"■ 7, ^f T [WW* + SjQA + \W l+N T D i W i+N 

8W { (1 + ct !+n V, +n ) 2 

3d 8d T (52) 

where a i+N = — —(AS i + (d i +g i )B i u i+N — ^ eJi tN ), and the tuning law for the i th actor 

dfy jeN t 

NNas 

A + n = -a i+N {(SiW i+N - F,<r£ N WJ) - jD t W I+N ?^<- W t 

4 m si 



4 ; .^J/ ' °" dSj ' " •' " ' dSj : m. 



where 



— 8d 1 j dd- t _ t 

D >i x ) = 7T B i R ti B i tt , m ={a i+N a i+N +l), cr i+N = cr i+N / (cr i+N cr i+N + 1) , and 

a, >0,...a j+N >0 and J^>0,G,->0, ieN are tuning parameters. 

Theorem 6. Online Cooperative Games. 

Let the error dynamics be given by (8), and consider the cooperative game formulation in 
(15). Let the critic NN at each node be given by (48) and the control input be given for each 
node by actor NN (51). Let the tuning law for the i th critic NN be provided by (52) and the 

tuning law for the i th actor NN be provided by (53). Assume °",+n = "'i+N / (°"i+n a i+N + 1) is 

persistently exciting. Then the closed-loop system states S t (t) , the critic NN errors VV t , and 

the actor NN errors W i+N are uniformly ultimately bounded. 

Proof: 

The proof is similar to (Vamvoudakis, 2011). 



Remark 1. Theorem 6 provides algorithms for tuning the actor/ critic networks of the N 
agents at the same time to guarantee stability and make the system errors S t (t) small and 
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the NN approximation errors bounded. Small errors guarantee synchronization of all the 
node trajectories. 

Remark 2. Persistence of excitation is needed for proper identification of the value functions 
by the critic NNs, and nonstandard tuning algorithms are required for the actor NNs to 
guarantee stability. It is important to notice that the actor NN tuning law of every agent 
needs information of the critic weights of all his neighbors, while the critic NN tuning law of 
every agent needs information of the actor weights of all his neighbors, 

Remark 3. NN usage suggests starting with random, nonzero control NN weights in (51) in 
order to converge to the coupled HJ equation solutions. However, extensive simulations 
show that convergence is more sensitive to the persistence of excitation in the control inputs 
than to the NN weight initialization. If the proper persistence of excitation is not selected, 
the control weights may not converge to the correct values. 

Remark 4. The issue of which inputs z,(£) to use for the critic and actor NNs needs to be 
addressed. According to the dynamics (8), the value functions (13), and the control inputs 
(16), the NN inputs at node i should consist of its own state, the states of its neighbors, and 
the costates of its neighbors. However, in view of (31) the costates are functions of the states. 
In view of the approximation capabilities of NN, it is found in simulations that it is suitable 
to take as the NN inputs at node i its own state and the states of its neighbors. 

The next result shows that the tuning laws given in Theorem 6 guarantee approximate 
solution to the coupled HJ equations (25) and convergence to the Nash equilibrium. 

Theorem 7. Convergence to Cooperative Nash Equilibrium. 

Suppose the hypotheses of Theorem 6 hold. Then: 

a. H i {S i ,W i ,«;,«_;), Vi e N are uniformly ultimately bounded, where 

u i = -j(d t + gj)Rjj Bj — '- Wj . That is, W i converge to the approximate cooperative 
dS { 

coupled HJ-solution. 

b. u i+N converge to the approximate cooperative Nash equilibrium (Definition 2) for 

every i . 

Proof: 

The proof is similar to (Vamvoudakis, 2011) but is done only with respect to the neighbors 
(local information) of each agent and not with respect to all agents. 

Consider the weights W t ,W j+N to be UUB as proved in Theorem 6. 

a. The approximate coupled HJ equations are H i {S i ,W i ,u ; ,m_ ; ), Vi e N . 

HXS l ,W l ,u,,u_ l )^HXS,,W l W_ l ) = SjQ l A + W^AS l -l(d l+ gfW l T ^B l R^B l T ^ W { 
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+t T (<*y +gi) 2 V^i r — BiRfRyRfB?-^- Wi+iW/4 V e B.RfB, 7 -^- W,-s H1 

4 ,6?, ^ 9 ^ d$ ^ ; J 7J 7 3<^ ; H/i 

where £ H , , Vj are the residual errors due to approximation. 
After adding zero we have 



h,.(^,w;. w_ ! ) = -w, r ^A<y, -±(4+** W -^b^b 



Ma , „ \2rfrT Wi „ „-l v T 80i py 



r^ ; 



<v,. 



r<) ; 



CO; 3d; 3d,- 3d, 



i E ( d , + g,) Z W, T — BiRtR«RtB? -i- W, + i V (d,- + ?,) 2 W, T — J-B.R^R ;,R- 1 B, t 

4 ^j v ; o;' 7 oe 7 77 y ;; ; q? ; 2 ^j v / &]' ; ^^ ; 77 y 77 / 



;sNj 









+7 Z K +^ ; ) 2 W ; r ^B ; RT.iR ,2^8,'— W, + W; T ^ y e^.RfB^-i- W,-e H] 

2 A^\ 1 &]> 7 Qg 7 77 y 77 7 5 c. 7 « 5J Z^ '7 7 77 7 9( y. 7 H/, 



;sNj 



jeN; 



_lW r ^^y eBR^B 7 —^- W-±W T ^-Y e-BR' 1 B T —L W 



s<y ; 



I 7'eNi 



3J 



rrf ; 



T 



2 ' xx *-* 1 J a J ax. J 



m, 



i 7'eN, 



OS: 



(54) 



But 



W i =-W i +W i , Vj. 
After taking norms in (55) and letting ||W,-|| < W jmax one has 



(55) 



\\W- = \\-W- +W\\<\\W\\ + \\W\\<\\W\\ + W- 

' ' ' — ' II Ml — M (max 



Now (54) with sup \\£ H i. < s i becomes 



H,.(<y i ,w,.w_ i )< Wj 



+iw+^) 2 IK 



3tf ; 



a|$|+t(4+s<) 2 NI 



d* 



rrf. 



II R II 2 II R- 1 II 

p. F" 



<•>,. 



<-v)' 



^■fNII+fw+^flK^i 



£<* 



b- ii 2 iik" 1 iirii w- II + w- ) 

II " III ' II imax I 
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+ i£ty + Sy) 2 HI 



i^ N , 



cS, 






5<>\. 



N rVW 



KH + ^w) 



rA 






<>/ 



?<5: 



(11^1+ w imax )+±|w;.| 



cS, 



s^HIII* 



jeN, 



94 



rt>, 



n 



+ 1 I|W- I 

2 1| 'rnax| 



d* 



?& 



1*1 h 



;ceN ; 



( \y,. 



(Iwll + w- ) 

1 / ;max J 



+ i('||W'-|| + W- ) 

2 \ ' [max I 



rd' ; 






r\>\. 



IW/max + ^2 



(56) 



All the signals on the right hand side of (56) are UUB and convergence to the approximate 
coupled HJ solution is obtained for every agent. 

b. According to Theorem 6, W, +N -W; L Vz are UUB. Then it is obvious that u i+N , Vi give 

the approximate cooperative Nash equilibrium (Definition 2). 

■ 

6. Simulation results 

This section shows the effectiveness of the online approach described in Theorem 6 for two 
different cases. 

Consider the three-node strongly connected digraph structure shown in Figure 1 with a 
leader node connected to node 3. The edge weights and the pinning gains are taken equal to 
1 so that d 1 = d 2 =l,d 3 =2 . 

1 




Fig. 1. Three agent communication graph showing the interactions. 
Select the weight matrices in (9) as 
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Oil _ Ql2 - «3 



1 

1 



, R u = 4, R 12 = 1, R-, 



-1, 



: -4, R ?? = 9, R ?3 = 1, -R,, = 9, £,, = 1, R, 



In the examples below, every node is a second-order system. Then, for every agent 



s i = \_ S il S i2 

According to the graph structure, the information vector at each node is 



2 i = \_ s l s i 



{Si 



*n 



-[s. 



f si si] 



Since the value is quadratic, the critic NNs basis sets were selected as the quadratic vector in 
the agent's components and its neighbors' components. Thus the NN activation functions 
are 



<fa(S 1 ,0,S 3 ) = [sf 1 5 n 8 n S\ 2 £ 3 2 j 5 3l S 32 S 32 
< t > i{Si>3 1 ,Q) = \S U S xl S 12 <^i2 ^21 ^21^22 ^21 



<f> 3 (S 1 ,S 2 ,S 3 ) = yS^ 1 S U S 12 S\ 2 S 21 S 21 S : 



22 "22 "31 "31"32 "32 



6.1 Position and velocity regulated to zero 

For the graph structure shown, consider the node dynamics 

'-2 1 

-4 -1 

and the command generator x 

The graphical game is implemented as in Theorem 6. Persistence of excitation was ensured 
by adding a small exponentially decreasing probing noise to the control inputs. Figure 2 
shows the convergence of the critic parameters for every agent. Figure 3 shows the evolution 
of the states for the duration of the experiment. 

6.2 All the nodes synchronize to the curve behavior of the leader node 

For the graph structure shown above consider the following node dynamics 
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Fig. 2. Convergence of the critic parameters. 
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Fig. 3. Evolution of the system states and regulation. 
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Fig. 4. Convergence of the critic parameters. 

The command generator is marginally stable with poles at s = ±j , so it generates a 
sinusoidal reference trajectory. 
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The graphical game is implemented as in Theorem 6. Persistence of excitation was ensured 
by adding a small exponential decreasing probing noise to the control inputs. Figure 4 
shows the critic parameters converging for every agent. Figure 5 shows the synchronization 
of all the agents to the leader's behavior as given by the circular Lissajous plot. 

7. Conclusion 

This chapter brings together cooperative control, reinforcement learning, and game theory 
to solve multi-player differential games on communication graph topologies. It formulates 
graphical games for dynamic systems and provides policy iteration and online learning 
algorithms along with proof of convergence to the Nash equilibrium or best response. 
Simulation results show the effectiveness of the proposed algorithms. 
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1. Introduction 

Nowadays, advanced control systems are playing a fundamental role in plant operations 
because they allow for effective plant management. Typically, advanced control systems 
rely heavily on real-time process modeling, and this puts strong demands on developing 
effective process models that, as a prime requirement, have to exhibit real-time responses. 
Because in many instances detailed process modeling is not viable, efforts have been 
devoted towards the development of approximate dynamic models. 

Approximate process models are based either on first principles, and thus require good 
understanding of the process physics, or on some sort of black-box modeling. Neural 
network modeling represents an effective framework to develop models when relying on an 
incomplete knowledge of the process under examination (Haykin, 2008). Because of the 
simplicity of neural models, they exhibit great potentials in all those model-based control 
applications that require real-time solutions of dynamic process models. The better 
understanding acquired on neural network modeling has driven its exploitation in many 
process engineering applications (Hussain, 1999). 

Genetic algorithms (GA) are model machine learning methodologies, which derive their 
behavior from a metaphor of the processes of evolution in nature and are able to overcome 
complex non-linear optimization tasks like non-convex problems, non-continuous objective 
functions, etc. (Michalewitz, 1992). They are based on an initial random population of 
solutions and an iterative procedure, which improves the characteristics of the population 
and produces solutions that are closer to the global optimum. This is achieved by applying a 
number of genetic operators to the population, in order to produce the next generation of 
solutions. GAs have been used successfully in combinations with neural and fuzzy systems 
(Fleming & Purhouse, 2002). 

Distillation remains the most important separation technique in chemical process industries 
around the world. Therefore, improved distillation control can have a significant impact on 
reducing energy consumption, improving product quality and protecting environmental 
resources. However, both distillation modeling and control are difficult tasks because it is 
usually a nonlinear, non-stationary, interactive, and subject to constraints and disturbances 
process. 
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In this scenario, most of the contributions that have appeared in literature about advanced 
control schemes have been tested for nonlinear simulation models (Himmelblau, 2008), 
while applications with advanced control algorithms over industrial or pilot plants (Frattini 
et al, 2000) (Varshney and Panigrahi, 2005) (Escano et al, 2009) or even with classical control 
(Noorai et al, 1999) (Tellez-Anguiano et al, 2009) are hardly found. 

Composition monitoring and composition control play an essential role in distillation control 
(Skogestad, 1997). In practice, on- line analyzer for composition is rarely used due to its costs 
and measurement delay. Therefore composition is often regulated indirectly using tray 
temperature close to product withdrawal location. In order to achieve the control purpose, 
many control strategies with different combination of manipulated variables configurations 
have been proposed (Skogestad, 2004). If a first-principles model describes the dynamics with 
sufficient accurately, a model-based soft sensor can be derived, such an extended Kalman filter 
or its adaptive versions (Venkateswarlu & Avantika, 2001), while inferential models can also 
be used when process data are available by developing heuristic models (Zamprogna et al, 
2005). Artificial neural networks can be considered from an engineering viewpoint, as a 
nonlinear heuristic model useful to make predictions and data classifications, and have been 
also used as a soft sensors for process control (Bahar et al, 2004). 

Nevertheless, few results are reported when is considered the composition control of 
experimental distillation columns, and some results are found either by applying direct 
temperature control (Marchetti et al, 1985) or by using the vapor-liquid equilibrium to 
estimate composition from temperature (Fileti et al, 2007), or even by using chromatographs 
(Fieg,2002). 

In this chapter we describe the application of adaptive neural networks to the estimation of 
the product compositions in a binary methanol-water continuous distillation column from 
available temperature measurements. This software sensor is then applied to train a neural 
network model so that a GA performs the searching for the optimal dual control law 
applied to the distillation column. The performance of the developed neural network 
estimator is further tested by observing the performance of the neural network control 
system designed for both set point tracking and disturbance rejection cases. 

2. Neural networks and genetic algorithms for control 

2.1 Neural networks for identification 

Neural networks offer an alternative approach to modelling process behaviour as they do 
not require a priori knowledge of the process phenomena. They learn by extracting pre- 
existing patterns from a data set that describe the relationship between the inputs and the 
outputs in any given process phenomenon. When appropriate inputs are applied to the 
network, the network acquires knowledge from the environment in a process known as 
learning. As a result, the network assimilates information that can be recalled later. Neural 
networks are capable of handling complex and nonlinear problems, process information 
rapidly and can reduce the engineering effort required in controller model development 
(Basheer & Hajmeer, 2000). 

Neural networks come in a variety of types, and each has their distinct architectural 
differences and reasons for their usage. The type of neural network used in this work is 
known as a feedforward network (Fig. 1) and has been found effective in many applications. 
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It has been shown that a continuous-valued neural network with a continuous differentiable 
nonlinear transfer function can approximate any continuous function arbitrarily well in a 
compact set (Cybenko, 1989). 




Fig. 1. Feedforward neural network architecture 

There are several different approaches to neural network training, the process of 
determining an appropriate set of weights. Historically, training is developed with the 
backpropagation algorithm, but in practice quite a few simple improvements have been 
used to speed up convergence and to improve the robustness of the backpropagation 
algorithm (Hagan & Menhaj, 1994). The learning rule used here is common to a standard 
nonlinear optimization or least-squares technique. The entire set of weights is adjusted at 
once instead of adjusting them sequentially from the output layer to the input layer. The 
weight adjustment is done at the end of each epoch and the sum of squares of all errors for 
all patterns is used as the objective function for the optimization problem. 

In particular we have employed the Levenberg-Marquardt algorithm to train the neural 
network used (Singh et al, 2007), which is a variation of the Newton's method, designed for 
minimizing functions that are sums of squares of other nonlinear functions. Newton's 
method for optimizing a performance index F(x) is given by 

x k+i — x k~ A k g k (1) 

where A k — V 2 F(x)\ x=Xk and g k = VF(x)\ x=x are the hessian and the gradient of F(x), 
respectively, and where x k is the set of net parameters at time k. In cases where F(x) is the 
sum of the square of errors e(x) over the Q targets in the training set 

F{x)=Y Q i=1 ef{x) = e T {x)e{x) (2) 

then the gradient would be given by 

VF(x) = 2J T (x)e(x) (3) 

where J(x) is the Jacobian matrix formed by elements — ■ — . On the other hand, the hessian 
would be approximated by 
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V 2 F(x) = 2f(x)-](x) (4) 

Then, substituting (3) and (4) into (1), it results in the Gauss-Newton method 

x k +i -x k - U T (x k )J(x k ) ]~ 1 J T (x k )e(x k ) (5) 

Adding a constant term ji k l to J T (x k )J(x k ), this lead to the Levenberg-Marquardt training 
rule so that 

x k +i -x k - U T (x k )](x k ) + \i k l ]~ 1 J T {x k )e(x k ) (6) 

where \x k is the learning coefficient, which is set at a small value in the beginning of the 
training procedure (p. k = le-03) and is increased (decreased) by a factor d > 1 (i.e. $ = 10) 
according to the increase (decrease) of F(x) in order to provide faster convergence. In fact, 
when ji k is set to a small value the Levenberg-Marquardt algorithm approaches that of 
Gauss-Newton, otherwise it behaves as a gradient descent technique. The neural network 
was configured to stop training after the mean squared error went below 0.05, the minimum 
gradient went below le-10 or the maximum number of epochs was reached (normally a high 
number is selected so that this is a non-limiting condition). 

The identification of the neural network model occurred via a dynamic structure constituted 
by a feedforward neural network representing the nonlinear relationship between input and 
output signals of the system to be modelled. The application of feedforward networks to 
dynamic systems modelling requires the use of external delay lines involving both input 
and output signals (Norgaard et al, 2000). 

The network input vector dimension was associated with the time window length selected 
for each input variable, which was dependent on distillation column dynamics and is 
usually chosen according to the expertise of process engineers (Basheer & Hajmeer, 2000). 
The hidden layer dimension was defined by using a trial and error procedure after selecting 
the input vector, while the net's output vector dimension directly resulted from the selected 
controlled variables. 

Therefore, the neural network identification model NN t after selecting the optimal input 
vector was given by 

x(t + 1) = NN,{x(t),u{t)) (7) 

where x (t + 1) stands for the predicted value of the neural network corresponding to the 
actual net input vector u(t) and the state vector x(t)_ 

The resulting identification model was obtained after selecting the best neural network 
structure among the possible ones, after a training process. Finally, a neural network 
validation process was performed by comparing the network output with additional data 
that were not included in the training data (validation set). 

2.2 Genetic algorithms for optimization and control 

Genetic Algorithms are adaptive methods which can be used to solve optimization 
problems. They are based on genetic processes of biological organisms. Over many 
generations, natural populations evolve according to the principles of natural selection and 
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survival of the fittest. In nature, individuals with the highest survival rate have relatively a 
large number of offspring, that is, the genes from the highly adapted or fit individuals 
spread to an increasing number of individuals in each successive generation. The strong 
characteristics from different ancestors can sometimes produce super-fit offspring, whose 
fitness is greater than that of either parent. In this way, species evolve to become better 
suited to their environment in an iterative way by following selection, recombination and 
mutation processes starting from an initial population. 

The control scheme here proposed is based on the different strengths that neural network 
and genetic algorithms present. One of the most profitable characteristic of the neural 
networks is its capability of identification and generalization while genetic algorithms are 
used for optimizing functions. 

If an accurate identification model is available, the controller can use the information 
provided by selecting the optimum input that makes the system as near as possible to the 
goal to achieve. So one of the main differences between this controller and the rest is the 
way it selects the inputs to the system. 




e*<— 



Fig. 2. Genetic Algorithm Structure 

In this way, the function to minimize in each step is the absolute value of the difference 
between the predicted output (by means of the neural identification network) and the 
reference. This difference depends, usually, on known variables as past states of the system 
and past inputs and on unknown variables as are the current inputs to apply. Those inputs 
will be obtained from the genetic algorithm. 



2.3 Neural networks for estimation 

Most popular sensors used in process control are the ones that measure temperature, 
pressure and fluid level, due to the high accuracy, fast response properties and their 
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cheapness. On the other hand, some of the most controlled variables, such as composition, 
present great difficulties in the measurement phase because it should be done off-line in 
laboratory, by involving both a high delay time and an extra cost due to the use of expensive 
equipment requiring both initial high investment and maintenance, such as occurs with 
chromatography. 

The composition control is crucial in order to achieve the final product specifications during 
the distillation process. The use of sensors able to infer composition values from secondary 
variables (values easier to be measured) could be a solution to overcome the referred 
drawbacks, being this approach defined as a software sensor (Brosilow & Joseph, 2002). 

In this way, an inferential system has been developed for achieving an on-line composition 
control. As the value of the controlled variable is inferred from other secondary variables, 
the model should be very accurate mainly in the operating region. The inferential system 
based on the first principles model approach presents the drawback of increasing 
computing time as the number of variables increase. 

A black-box model approach relating the plant outputs with the corresponding sampled 
inputs has been used instead. Neural networks have proven to be universal approximators 
(Haykin, 2008), so they will be used to infer the composition from other secondary variables, 
defining thus the neural soft estimator. 

One of the main difficulties in determining the complete structure of the neural estimator is 
the choice of the secondary variables to be used (both the nature and the location), selected 
among the ones provided by the set of sensors installed on the experimental pilot plant. In 
the literature there are several papers dedicated to the selection of variables for composition 
estimation and no consensus is reached in terms of number or position of the secondary 
sensors (here position is understood as the stage or plate where the variable is measured). In 
(Quintero-Marmol et al, 1991), the number that assures robust performance is N c + 2, where 
N c is the number of components. With respect to the location of the most sensitive trays, 
(Luyben, 2006) develops a very exhaustive study and concludes that the optimal position 
depends heavily on the plant and on the feed tray. In this way, the neural estimator should 
have as an input the optimum combination of selected secondary variables to determine 
accurately the product composition. 

In order to select the most suitable secondary variables for our control purposes, a 
multivariate statistical technique based on the principal component analysis (PCA) 
methodology (Jackson,1991) has been used, following the same approach described by 
(Zamprogna et al,2005). The resulting neural network estimator NN E is given by 

x p (t) = NN E (x s (t)) (8) 

where x p (t) and x s (t) stands for the primary and secondary selected variables. 

2.4 Neurogenetic control structure 

As an accurate neural network model that relates the past states, current states, and the 
current control inputs with the future outputs is available, the future output of the system 
can be predicted depending on the control inputs through a non linear function. In this way, 
the function to be minimized in each step is a cost function that is related to the absolute 
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value of the difference between the predicted output and the desired reference to follow. 
This difference depends, usually, on known variables such as past inputs and past states of 
the system and on unknown variables such as the current control inputs to apply, which 
will be obtained from the genetic algorithm. 

In this way, the optimization problem for controlling the distillation plant can be stated as 
the problem of finding the input that minimizes the norm of the difference, multiplied by a 
weighting matrix between the reference command to follow and the neural network model 
output, considering the input and the past and current states of the system. This procedure 
can be stated as min \\K W ■ (x r — NNi(x, u))\\, with x r representing the reference command to 
follow, NN j is the neural network model output, x represents the past and current states of 
the system, u G U is the control action and U is the universe of possible control actions and 
K w is a weighting matrix. 

In the present case, the reference command x r will be given by the desired composition 
variables together with the desired level variables, while u £ U represents the optimum 
neurogenetic control action, and the weighting matrix penalizes the errors in composition 
twice the errors in level, since composition control is more difficult to achieve than level 
control. In Fig. 3 the neurogenetic control strategy that is used here is shown, together with 
the neural composition estimator. 
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Fig. 3. Neural Estimation and Neurogenetic Control Structure 



3. Application to a pilot distillation column 

3.1 Description of the pilot distillation column 

The pilot distillation column DELTALAB is composed of 9 plates, one condenser, and one 
boiler (Fig. 4). The instrumentation equipment consists of 12 Pt 100 RTD temperature 
sensors (T1-T12), 3 flow meters (FI1-FI3), 2 level sensors (LT1-LT2) and 1 differential 
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pressure meter (PD), together with 3 pneumatic valves (LIC1-LIC2-TIC2) and a heating 
thermo-coil (TIC1), with up to four control loops for plant operation. Additionally, feed 
temperature and coolant flow control are included with corresponding valve (FIC1) and 
heating resistance (PDC1), being both variables considered as disturbances. 




Fig. 4. Pilot distillation plant configuration 

The condenser provides the necessary cooling to condense the distilled product. The 
condenser contains the cooling water provided by an external pump. The flow of the cooling 
liquid is regulated through a pneumatic valve with one flow controller, which as a last 
resort depends on the variable water flow supply. Two temperature sensors measure the 
temperature of the inlet and outlet flows. 

Once the top stream is condensed, the liquid is stored in an intermediate reflux drum, 
endowed with level meter, temperature sensor and recirculation pump for reflux stream. 
The reflux to distillate ratio is controlled by 2 proportional pneumatic valves for reflux and 
distillate respectively, each flow measured through the corresponding flow meter with 
display. 
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The main body of the distillation column is composed of 9 bubble cap plates distributed into 
3 sections. Two of them are connected to the feeding device, and can either function like 
feeding or normal plates, selecting each one through a manual valve. Four temperature 
sensors measure the temperature in each section junction. 

The boiler provides the required heat to the distillation column by actuating on an electric 
heating thermo-coil located inside the boiler. A temperature sensor is located inside the 
boiler and a level meter measures the liquid stored in an intermediate bottom drum. A 
differential-pressure sensor indicates the pressure changes throughout the column which is 
operated at atmospheric pressure. The bottom flow is controlled by a proportional 
pneumatic valve and two temperature sensors measure the temperature of the inlet and 
outlet flows before cooling, with corresponding flow meters with display. 

The feeding ethanol-water mixture is stored in a deposit, whose temperature is controlled 
by a pre-heating electric thermo-coil. The mixture to be distilled is fed into the column in 
small doses by a feeding pump with temperature controller (TIC3) and sensors installed to 
measure the temperature of the inlet and outlet feed flows. 

The whole instrumentation of the distillation pilot plant is monitored under Lab VIEW 
platform and is connected to the neural based controller designed under MATLAB platform, 
through a communication system based both on PCI and USB buses, with up to four control 
loops. In this experimental set-up, boiler heat flow Qb, reflux valve opening Vr, distillate 
valve opening Vd and bottom valve opening Vg constitute the set of manipulated variables, 
while light composition Co, bottom composition Cb, light product level Ld and heavy 
product level Lb define the corresponding set of controlled variables (Fig. 5), while the feed 
flow temperature Tf is considered as a disturbance. 
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Fig. 5. Pilot distillation plant configuration 
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It is important to highlight that a dynamical model has not been derived to represent the 
pilot column behavior, instead of this we have made use of an approximate neural network 
model to identify the plant dynamics starting from selected I/O plant data operation. 

3.2 Monitoring and control interface system 

The monitoring and control interface system requires a communication system between the 
sensors and actuators on the one hand and the computer on the other hand throughout I/O 
modules, whose specifications are settled by the instrumentation characteristics utilized 
(Table 1 and 2). 

In order to manage the I/O signals, USB and PCI buses have been chosen. On the one hand, 
the PCI bus enables the dynamic configuration of peripheral equipments, since during the 
operating system startup, the devices connected to PCI buses communicate with the BIOS 
and calculate the required resources for each one. On the other hand, the USB bus entails a 
substantial improvement regarding the 'plug and play' technology, having as main objective 
to suppress the necessity of acquiring different boards for computer ports. Besides this, an 
optimal performance is achieved for the set of different devices integrated into the 
instrumentation system, connectable without the needing to open the system. 



Sensor 


Variable 


Physical 

Range 


Magnitude 


Signal Range 


Measuring 
Accuracy 


T1-T12 


Temperature 


-200-119 °C 


Resistance 


18.5-145.7 _ 


0.01 °c 


FI1-FD 


Flowrate 


0-5 1/h 


Current 


4-20 mA 


= 2.5 % 


LT1 


Level 


(M95 mm 


Current 


4-20 mA 


= 0.075% 


LT2 


Level 


0-950 mm 


Current 


4-20 mA 


= 0.075% 


PD 


Diff Pressure 


0-25 mbar 


Current 


4-20 mA 


= 0.075% 



Table 1. Sensors characteristics for the pilot distillation column 

The acquisition system configuration for the monitoring and control of the pilot plant is 
constituted by the next set of DAQ (Data Acquisition) boards: NI PCI-6220, NI-PCI-6722, NI- 
USB-6009, NI-USB-6210 for analog voltage signal acquisition and NI-PCI-6704 for analog 
current signal acquisition, all supplied by National Instruments (NI). Measurements 
obtained from the sensors have been conditioned to operate into the standard operational 
range, and signal averaging for noise cancelation has been applied using specific LabVIEW 
toolkits (Bishop, 2004). 

The monitoring and control interface system developed for the pilot plant is configured 
throughout the interconnection of the NI Data acquisition system with both the LabVIEW 
monitoring subsystem and the neurogenetic controller implemented in MATLAB (Fig. 6), 
both environments linked together through the Mathscripts and running under a Intel core 
duo with 2.49 GHZ and 3 GB of RAM. 
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Control Loop 


Actuator 


Actuation Type 


Magnitude 


Signal Range 


PDCl 


Resistance 


On/Off 


Voltage 


0-5 V 


TIC1 


Resistance 


On/Off 


Voltage 


0-5 V 


TIC2 


Valve 


Proportional 


Current 


4-20 mA 


LIC1 


Valve 


Proportional 


Current 


4-20 mA 


LIC2 


Valve 


Proportional 


Current 


4-20 mA 


FIC1 


Valve 


Proportional 


Current 


4-20 mA 



Table 2. Actuators characteristics for the pilot distillation column 
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Fig. 6. Monitoring and control interface for pilot distillation plant 

The process control scheme developed in each operation cycle implies the execution of five 
different actions: system initializing, buttons control reading from VI (Virtual Instruments), 
reading plant data from instruments, control action calculation and writing control data to 
instruments. 



3.3 Neural composition estimator and neurogenetic controller 

The complete controlled system is composed of a neural network model of the process and a 
control scheme based on a genetic algorithm which utilizes both the composition and the 
level variables to get the quasi-optimal control law, by using the neural composition 
estimator (Fig. 3) for both determining and monitoring the composition of light and heavy 
components from secondary variable measurements. 

After applying the selection method, the inputs to the neural estimation network turned out 
to be four secondary variables, namely, three temperatures T 6 , T s , T 2 , each corresponding to 
reflux, top and bottom temperatures, and differential pressure drop DPD, while Co and Cb 
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compositions were the net outputs. This structure is in line with what the literature suggests 
(Quintero-Marmol et al, 1991) (Zamprogna et al, 2005) in terms of both the number of the 
selected measurements and its distribution. This fact contrasts with the standard approach 
consisting in selecting two temperatures for a two composition estimation (Medjell and 
Skogestad,1991) (Strandberg and Skogestad,2006). However, this assumption is not possible 
when the vapor-liquid equilibrium has a strong nonlinear behavior (Baratti et al.,1998) 
(Oisiovici and Cruz, 2001), so that holding the temperature constant does not imply that 
composition will also be constant (Rueda et al, 2006). 

The final network structure selected for the neural composition estimator was a 4-25-2 net, 
trained using the Levenberg-Marquardt algorithm (Hagan et al, 2002), with a hidden layer 
configuration selected after a trial and error process and input layer determined by the PCA 
based algorithm for selection of the secondary variables previously exposed. 

The training data set used herein consisted of 700 points collected randomly from a whole 
data set of more than 27000 acquired points, all obtained from several experiments carried 
out with the pilot distillation column by covering the whole range of operation. A different 
subset of 700 points has been also used for validation. For this purpose we have analyzed 
several samples of an ethanol-water mixture during the separation process by using a flash 
chromatograph VARIANT, and the composition error mean obtained was lower than 1.5%. 

The final network structure selected for the neural plant model was a 22-25-6 neural 
feedforward architecture trained by using the Levenberg-Marquardt algorithm and 
validated throughout the set of I/O experimental data. The hidden layer configuration was 
selected after the algorithm as it was stated in the previous section, using this time Vr, Vd, 
Vb, Qb, T2, Ts, Tfo Tf, Lq, Lg, and DPD delayed values as inputs, while T2, Ts, Tg, DPD, Lq, Lb 
were the estimated outputs. The neural net was trained with a different subset of 750 points 
selected randomly from the whole data set of 27000 acquired points with sampling T = 2 s, 
both by using a PID analog control module, by changing set-points for each of the controlled 
variables into its operating range and by working on open loop conditions. The neural net 
was also validated with another subset of 750 points comparing its outputs to the real 
system's outputs in independent experiments. 

The neurogenetic controller is characterized by a population of 75 inhabitants, 50 generations 
and a codification of 8 bits. The maximum is accepted if it is invariant in 5 iterations. All these 
parameters were estimated for achieving a time response lower than 1.3 seconds for the 
computational system used for controlling the experimental distillation plant. 

3.4 Results 

In order to test the validity of the proposed control scheme, the performance of the 
neurogenetic control strategy is compared against a PID control strategy by using four 
decoupled PID controllers relating Vr, Qb, Vd and Vb manipulated variables with the 
corresponding controlled variables Co, Cb, Lq and Lg. Obviously in order to compare properly 
both strategies, the PID approach should control the same variables, in a way the composition 
is indirectly controlled, by following the standard LV configuration (Skogestad, 1997). The PID 
parameters set selected for each controlled variable has been heuristically tuned according to 
the analog PID values set by the DELTALAB field expert when the pilot column is supplied. 

Several changes in composition set points on top and bottom purity have been made to test 
the neurogenetic controller performance (Fig. 7). As it is shown, the system is able to reach 
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the required references in composition but is a bit slow in its response. The response 
obtained with the PID approach presents a bigger settling time and overshoot and a poorer 
response to changes in the targets in the coupled variables. In fact, the ISE (integral square 
error) which characterizes the accuracy of both control schemes during tracking of reference 
commands, is significantly lower for the neurogenetic control as compared to the PID 
control both controlled variables, with a ISE d PID = 4719.9 (%) 2 • s, ISE dNeuroGA — 
3687.2 (%) 2 ■ s for top composition and lSE b PID — 2427.6 (%) 2 ■ s, ISE bNeuroGA = 
2071.8 (%) 2 ■ s for bottom composition respectively. These facts imply a better performance 
even when changing conditions are present (variable feed changes), due to the adaptive 
nature of the neurogenetic controller. 

In Fig. 8 are displayed the changes in control actions Vr, Vp, Vh (in % of opening) and Qb (in % 
of maximum power) corresponding to the set point changes on top and bottom composition as 
described formerly for the neurogenetic control scheme. It must be emphasized that all control 
signal are within the operating range with minimum saturation effects, mainly due to mild 
conditions imposed to the time response profile during the neurogenetic design. 
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Fig. 7. Response of top and bottom composition for set point changes in ethanol purity in (a) 
60-70 % range on top (b) 5-12 % range on bottom for pilot distillation column under 
decoupled PID and neurogenetic control 
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Fig. 8. Control actions Vr, Vp, Vh and Qb ( a )-(d) for set point changes in ethanol purity in 60- 
70 % range on top and 5-12 % range on bottom for pilot distillation column under 
neurogenetic control. 



4. Conclusions 

Adaptive neural networks have been applied to the estimation of product composition 
starting from on-line secondary variables measurements, by selecting the optimal net input 
vector for estimator by using PCA based algorithm. Genetic algorithms have been used to 
derive the optimum control law under MATLAB, based both on the neural network model 
of the pilot column and the estimation of composition. This neurogenetic approach has been 
applied to the dual control of distillate and bottom composition for a continuous ethanol 
water nonlinear pilot distillation column monitored under Lab VIEW. 

The proposed method gives better or equal performances over other methods such as fuzzy 
or adaptive control by using a simpler design based exclusively on the knowledge about the 
pilot distillation column in form of I/O operational data. It is also necessary to highlight the 
potential benefits of artificial neural networks combined with GA when are applied to the 
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multivariable control of nonlinear plants, with unknown first-principles model and under 
an experimental set-up as was demonstrated with the distillation pilot plant. 

Future work is directed toward the application of this methodology to industrial plants and 
also towards the stability and robustness analysis due to uncertainty generated by the 
neural network identification errors when the plant is approximated. 
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1. Introduction 

The history of linear matrix inequalities (LMIs) in the analysis of dynamical systems dates 
from over 100 years. The story begins around 1890 when Lyapunov published his work 
introducing what is now called the Lyapunov's theory (Boyd et al., 1994). The researches 
and publications involving the Lyapunov's theory have grown up a lot in recent decades 
(Chen, 1999), opening a very wide range for various approaches such as robust stability 
analysis of linear systems (Montagner et al., 2009), LMI optimization approach (Wang et al., 
2008), H 2 (Apkarian et al., 2001; Assuncao et al., 2007a; Ma & Chen, 2006) or Hco (Assuncao 
et al., 2007b; Chilali & Gahinet, 1996; Lee et al., 2004) robust control, design of controllers for 
systems with state feedback (Montagner et al., 2005), and design of controllers for systems 
with state-derivative feedback (Cardim et al., 2009). The design of robust controllers can also 
be applied to nonlinear systems. 

In addition to the various current controllers design techniques, the design of robust 
controllers (or controller design by quadratic stability) using LMI stands out for solving 
problems that previously had no known solution. These designs use specialized computer 
packages (Gahinet et al., 1995), which made the LMIs important tools in control theory. 

Recent publications have found a certain conservatism inserted in the analysis of quadratic 
stability, which led to a search for solutions to eliminate this conservatism (de Oliveira et al., 
1999). Finsler's lemma (Skelton et al., 1997) has been widely used in control theory for the 
stability analysis by LMIs (Montagner et al., 2009; Peaucelle et al., 2000), with better results 
than the quadratic stability of LMIs, but with extra matrices, which allows a certain relaxation 
in the stability analysis (here called extended stability), by obtaining a larger feasibility region. 
The advantage found in its application to design of state feedback is the fact that the synthesis 
of gain K becomes decoupled from Lyapunov's matrix P (Oliveira et al., 1999), leaving 
Lyapunov's matrix free as it is necessarily symmetric and positive defined to meet the initial 
restrictions. 

The reciprocal projection lemma used in robust control literature H2 (Apkarian et al., 2001), 
can also be used for the synthesis of robust controllers, eliminating in a way the existing 
conservatism, as it makes feasible dealing with multiple Lyapunov's matrices, as in the case of 
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extended stability point, allowing extra matrices through a relaxation in the case of extended 
stability, making feasible a relaxation in the stability analysis (here called projective stability) 
through extra matrices. The synthesis of the controller K is depending now on an auxiliary 
matrix V, not necessarily symmetrical, and in this situation it becomes completely decoupled 
from Lyapunov's matrix P, leaving it free. 

Two critical points in the design of robust controllers are explored here. One of them 
is the magnitude of the designed controllers that are often high, affect their practical 
implementation and therefore require a minimization of the gains of the controller to facilitate 
its implementation (optimization of the norm of JQ.The other one is the fact that the system 
settling time can be larger than the required specifications of the project, thus demanding 
restrictions on LMIs to limit the decay rate, formulated with the inclusion of the parameter 7 
in LMIs. 

The main focus of this work is to propose new methods for optimizing the controller 's norm, 
through a different approach from that found in (Chilali & Gahinet, 1996), and compare it 
with the optimization method presented in (Assuncao et al., 2007c) considering the different 
criteria of stability, aiming at the advantages and disadvantages of each method, as well as the 
inclusion of a decay rate (Boyd et al., 1994) in LMIs formulation. 

In (Siljak & Stipanovic, 2000) an optimization of the controllers's norm was proposed for 
decentralized control, but without the decay rate, so no comparisons were made with this 
work due to the necessity to insert this parameter to improve the performance of the system 
response. 

The LMIs of optimization that will be used for new design techniques, had to be reformulated 
because the matrix controller synthesis does not depend more on a symmetric matrix, a 
necessary condition for the formulation of the existing LMI optimization. Comparisons will 
be made through a practical implementation in the Quanser's 3-DOF helicopter (Quanser, 
2002) and a general analysis involving 1000 randomly generated polytopic uncertain systems. 

2. Quadratic stability of continuous time linear systems 

Consider (1) an autonomous linear dynamic system without state feedback. Lyapunov proved 
that the system 

x(t) = Ax(t) (1) 

with x(t) £ K" e A 6 jr»x« a known matrix, is asymptotically stable (i.e., all trajectories 
converge to zero) if and only if there exists a matrix P = P 6 R" T " such that the LMIs (2) and 
(3) are met (Boyd et al., 1994). 

A'P + PA<0 (2) 

P > (3) 

Consider in equation (2) that A is not precisely known, but belongs to a politopic bounded 
uncertainty domain A. In this case, the matrix A within the domain of uncertainty can be 
written as convex combination of vertexes A, , j = 1, ..., N, of the convex bounded uncertainty 
domain (Boyd et al., 1994), i.e. A(ix) e A and A shown in (4). 

N N 

A = {A(a) € R" x " : A(a) = V ocjAj , V ctj = 1 , ctj > , ;' = 1...N} (4) 



N 


N 






L«i A j ■ 


■ L>/ 


= 1 , 


cij > 


y=i 


7=1 
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A sufficient condition for stability of the convex bounded uncertainty domain A (now on 
called polytope) is given by the existence of a Lyapunov's matrix P = P G R" x " such that 
the LMIs (5) and (6) 

A{oL)'P + PA{a) <0 (5) 

P > (6) 

are checked for every A(u) G A (Boyd et al v 1994). This stability condition is known as 
quadratic stability and can be easily verified in practice thanks to the convexity of Lyapunov's 
inequality that turns the conditions (5) and (6) equivalent to checking the existence of P = 
P' G 5R" X " such that conditions (7) and (8) are met with/ = 1,...,N. 

A'jP + PAj < (7) 

P > (8) 

It can be observed that (5) can be obtained multiplying by a, > and adding in j of ) ' = 1 to 

j = N. 

Due to being a sufficient condition for stability of the polytope A, conservative results are 
generated, nevertheless this quadratic stability has been widely used for robust controllers's 
synthesis. 

3. Decay rate restriction for closed-loop systems 

Consider a linear time invariant controllable system described in (9) 

x(t) = Ax(t) + Bu(t), x(0)=x (9) 

with A G R" x ", B G R" xm the matrix of system input, x(t) G R" the state vector and u(t) G 
R'" the input vector. Assuming that all state are available for feedback, the control law for the 
same feedback is given by (10) 

u{t) = -Kx(t) (10) 

being K G ^nixn a cons t an t elements matrix. Often the norm of the controller K can be 
high, leading to saturation of amplifiers and making the implementation in analogic systems 
difficult. Thus it is necessary to reduce the norm of the controllers elements to facilitate its 
implementation. 

Considering the controlled system (9) - (10), the decay rate (or largest Lyapunov's exponent) 
is defined as the largest positive constant 7, such that (11) 

lim e T f | W ()||=0 (11) 

t— >co 

remains for all trajectories x(t ), f > 0. 

From the quadratic Lyapunov's function (12), 

V(x(t))=x(t)'Px(t) (12) 

to establish a lower limit on the decay rate of (9), with (13) 

V(x(t))<-2 7 V(x(t)) (13) 
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for all trajectories (Boyd et al., 1994). 
From (12) and (9), (14) can be found. 

V(x(t)) =x{t)'Px(t)+x(t)'Px(t) 

= x(t)'(A - BK)'Px(t) + x(t)'P{A - BK)x(t) (14) 

Adding the restriction on the decay rate (13) in the equation (14) and making the appropriate 
simplifications, (15) and (16) are met. 

(A-BK)'P + P{A-BK) < -2jP (15) 

P > (16) 

As the inequality (15) became a bilinear matrix inequality (BMI) it is necessary to perform 
manipulations to fit them back into the condition of LMIs. Multiplying the inequalities (17) 
and (18) on the left and on the right by P , making X = P and G = KX results: 

AX-BG + XA' - G'B' + 2jX < (17) 

X > (18) 

If the LMIs (17) and (18) are feasible, a controller that stabilizes the closed-loop system can be 
given by K = GX . 

Consider the linear uncertain time-invariant system (19). 

x(t) = A(a)x{t) + B{a)u(t) (19) 

This system can be described as convex combination of the polytope's vertexes shown in (20). 

r r 

x(t) = £ ctjAjx(t) + £ KjBju(t) (20) 

with A and B belonging to the uncertainty polytope (21) 

N N 

{A,B) = {(A,B)(a) e R" xn : {A,B)(a) = ^^B^n, = l,a ; - > 0,/=l...N} (21) 

7=1 7=1 

being r the number of vertexes (Boyd et al., 1994). 

Knowing the existing theory for uncertain systems, Theorem 3.1 theorem can be enunciated 
(Boyd et al., 1994): 

Theorem 3.1. A sufficient condition which guarantees the stability of the uncertain system (20) 
subject to decay rate y is the existence of matrices X = X' 6 R" x " and G € R mx ", such that 
(22) and (23) are met. 

AjX - BjG + XA'j - G'B'j + 2jX < (22) 

X > (23) 

with j = 1, ...,r. 

When the LMIs (22) and (23) are feasible, a state feedback matrix which stabilizes the system can be 
given by (24). 

K = GX' 1 (24) 
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Proof. The proof can be found at (Boyd etal., 1994). □ 

Thus, it can be feedback into the uncertain system shown in (19) being (22) and (23) sufficient 
conditions for the polytope asymptotic stability, now for a closed-loop system subject to decay 
rate. 

4. Optimization of the K matrix norm of the closed-loop system 

In many situations the norm of the state feedback matrix is high, precluding its practical 
implementation. Thus Theorem 4.1 was proposed in order to limit the norm of K (Assuncao 
et al., 2007c; Faria et al., 2010). 

Theorem 4.1. Given an fixed constant ug > 0, that enables to find feasible results, it can be obtained a 
constraint for the K £ ]R mx » matrix norm from the state feedback, with K = GX _1 , X = X' > 6 



re- 
value for & can be found solving the optimization problem with the LMIs (25), (26) and (27). 



R" x " and G € R m x " finding the minimum value B, B > such that KK' < -^jl m . The optimum 



min/3 

'film G 

s.t. 



> 



(25) 



. a In. 

X > M> (26) 

AjX - BjG + XA'j - G'B'j + 2jX < (27) 

where I m and l n are the identity matrices ofm and n order respectively. 

Proof. The proof can be found at (Assuncao etal., 2007c). □ 

5. New optimization of the K matrix norm of the closed-loop system 

It can be verified that the LMIs given in Theorem 4.1 can produce conservative results, so in 
order to find better results, new methodologies are proposed. 

Using the theory presented in (Assuncao et al., 2007c) for the optimization of the norm of 
robust controllers subject to failures, it is proposed an alternative approach for the same 
problem grounded in Lemma (5.1). 

The approach of the optimum norm used was modified to fit to the new structures of LMIs that 
will be given in sequence. At first, this new approach has produced better results comparing 
to the existing ones for the optimization stated in Theorem 4.1 using the set of LMIs (22) and 
(23). 

Lemma 5.1. Consider L G ]R nxm a a given matrix and f> g R, & > 0. The conditions 

1. L'L<BI m 

2. LL' < BI n 
are equivalent. 
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Proof. Note that if L = the lemma conditions are verified. Then consider the case where 
L^O. 

Note that in the first statement of the lemma, (28) is met 

L'L < BI m ^ x'(L'L)x < Bx'x (28) 

for all x e W. 

Knowing that (29) is true 

x'(L'L)x < A max (L'L)x'x (29) 

and \ max (L'L) the maximum eigenvalue of L'L, which is real (every symmetric matrix has 
only real eigenvalues). Besides, when x is equal to the eigenvector of L'L associated to the 
eigenvalue A„,„(L'L), and x'(L'L)x = \ max {L'L)x'x. Thus, from (28) and (29), B > \ max {L'L). 

Similarly, for every z 6 R", the second assertion of the lemma results in (30). 

LL' < BI n ^ z'{LL')z < A max {LL')z'z < Bz'z (30) 

and then, B > \ max (LL'). 

Now, note that the condition (31) is true (Chen, 1999). 

A"'det(M n - L'L) = \ n det{M m - LL') (31) 

Consequently, every non-zero eigenvalue of L'L is also an eigenvalue of LL' . Therefore, 
\ max (L'L) = \ max (LL'), and from (29) and (30) the lemma is proved . □ 

Knowing that P = X -1 is the matrix used to define Lyapunov's quadratic function, Theorem 
5.1 is proposed. 

Theorem 5.1. Given a constant fig > 0, a constraint for the state feedback K g ]R mx " matrix norm 
is obtained, with K = GX' 1 , X = X' > 0, X e R" x " and G e R mxn by finding the minimum of 



ft, /3 > such that K'K < -£-In- You can get the minimum f> solving the optimization problem with 



I 
the LMIs (32), (33) and (34) 

min i 

"X G' 
s.t. 



> 



(32) 



Gpl„ 

X > fl I n (33) 

AjX - BjG + XA'j - G'B'j + 2jX < (34) 

where l m and I„ are the identity matrices ofm and n order respectively. 

Proof. Applying the Schur complement for the first inequality of (32) results in (35). 

BI m > e X - G'{BI m )- l G > (35) 
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Thus, from (35), (36) is found. 



X > \g'G => G'G < BX (36) 



Replacing G = KX in (36) results in (37) 

XK'KX < BX => K'K < BX' 1 (37) 

So from (33), (37) and (33), (38) is met. 

K'K < -£■ I n (38) 

f'o 

on which K is the optimal controller associated with (22). □ 

It follows that minimizing the norm of a matrix is equivalent to the minimization of a /3 > 
variable such that K'K < ^-I n , with Uq > 0. Note that the position of the transposed matrix 
was replaced in this condition, comparing to that used in Theorem 4.1. 

A comparison will be shown between the optimization methods, using the robust LMIs with 
decay rate (22) and (23) in the results section. Since the new method may suit the relaxed 
LMIs listed below, it was used in the comparative analysis for the control design for extended 
stability and projective stability. 

Finsler's lemma shown in Lemma (5.2) can be used to express stability conditions referring 
to matrix inequalities, with advantages over existing Lyapunov's theory (Boyd et al., 1994), 
because it introduces new variables and generate new degrees of freedom in the analysis of 
uncertain systems with the possibility of nonlinearities elimination. 

Lemma 5.2 (Finsler). Consider w G R">, £ G R"**"* and B G R'"* XH * with rank(B) < n x e B 1 - 
a basis for the null space of B (i.e., BB^ = 0). Then the following conditions are equivalent: 

1. iv'Civ < 0, V w ^ : Bw = 

2. B^CB 1 < 

3. Bfi € R : C - uB'B < 

4. 3XeR n ' xm " : C + XB + B'X' <Q 

Proof. Finsler's lemma proof can be found at (Oliveira & Skelton, 2001; Skelton et al., 1997). 

□ 

5.1 Stability of systems using Finsler's lemma restricted by the decay rate 



Consider the closed-loop system (9). Defining w 



[(A-BK) -I] , B x 



I 

{A-BK) 



and C 



2-yP P 
P 



Note that Bw = corresponds to (9) and w'Czo < 



corresponds to stability constraint with decay rate given by (12) and (13). In this case the 
dimensions of the lemma's variables (5.2) are: n x = 2n and m x = n. Considering that P is 
the matrix used to define the quadratic Lyapunov's function (12), the properties 1 and 2 of 
Finsler's lemma can be written as: 
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1. 3P = P' > such that 



27PP 
P 



<0Vx,x ^0 : [(A-BK) -I] 



2. 3P = P' > such that 
27PP 
P 



I 

{A-BK) 



I 

{A-BK) 



<0 



which results in the equations of stability, according to Lyapunov, including decay rate: 

1. x(tyPx{t)+x(tyPx(t)+2yx(tyPx(t) <0Vx,i^0 : x(t) = (A - BK)x(t) 

2. P(A - BK) + (A- BK)'P + 2-yP < 

Thus, it is possible to characterize stability through Lyapunov's quadratic function (V(x(t)) = 
x(t)'Px(t)), generating new degrees of freedom for the synthesis of controllers. 

From Finsler 's lemma proof follows that if the properties 1 and 2 are true, then properties 3 
and 4 will also be true. Thus, the fourth propriety can be written as (39). 

4. 3X G R 2 " x ", P = P' > such that 



27PP 
P 



+ X [(A-BK) -I] 



[A - BK) 
-I 



X' <0. 



(39) 



Choosing conveniently the matrix of variables X 



Z 
aZ 



with Z S R" x " invertible and not 



necessarily symmetric and a > a fixed relaxation constant of the LMI (Pipeleers et al., 2009). 

\Z~ l 
Developing the equation (39) and applying the congruence transformation 



left and 



z- 1 
z- 1 



on the right, is found (40). 



AZ'- 1 +Z- 1 A'-BKZ'- 1 -Z- 1 K'B>+2-yZ- 1 PZ'- 1 Z^PZ'^+aZ^A' '-aZ^K'l 
Z~' i PZ'- 1 +aAz'-' i -aBKZ'- 1 -Z- 1 -aZ'" 1 -nZ" 1 



Z" 



< 



on the 



Making Y = Z 
decay rate 7 



'-1. 



G = KY and Q = Y'PY, there were found LMIs (40) and (41) subject to 



AY + Y'A' -BG- G'B' + 2yQ Q + aY'A' - aG'B' - Y 
Q + aAY-aBG-Y' -aY-aY' 

Q>0 



<0, 



(40) 
(41) 



with Y G R" x ", Y ^Y',Ge R mx " and Q G R" x ", Q = Q' > 0, for some a > 0. 



These LMIs meet the restrictions for the asymptotic stability (Feron et al., 1996) of the system 
described in (9) with state feedback given by (10). It can be checked that the first principal 
minor of the LMI (40) has the structure of the result found in the theorem of quadratic stability 
with decay rate (Faria et al., 2009). Nevertheless, there is also, as stated in the Finsler's 
lemma, a greater degree of freedom because the matrix of variables Y , responsible for the 
synthesis of the controller, doesn't need to be symmetric and the Lyapunov's matrix now 
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turned into Q, which remains restricted to positive definite, is partially detached from the 
controller synthesis, since that Q = Y'PY. 

The stability of the LMIs derived from Finsler's lemma stability is commonly called extended 
stability and it will be designated this way now on. 



5.2 Robust stability of systems using Finsler's lemma restricted by the decay rate 

As discussed for the condition of quadratic stability, the stability analysis can be performed 
for a robust stability condition considering the continuous time linear system as a convex 
combination of r vertexes of the polytope described in (20). The advantage of using the 

Finsler's lemma for robust stability analysis is the freedom of Lyapunov's function, now 

r r 

defined as Q(a) = E &jQi, E a j = 1/ oti > e j = \...r, i.e., it can be defined a Lyapunov's 

function Q; for each vertex j. As Q(a) depends on a, the Lyapunov matrix use fits to 
time-invariant polytopic uncertainties, being permitted rate of variation sufficiently small. To 
verify this, Theorem 5.2 is proposed. 

Theorem 5.2. A sufficient condition which guarantees the stability of the uncertain system (20) is the 
existence of matricesY E R" x ", Q € R" x ", Qj = Q,' > e G € R mx ", decay rate greater than y 
and a fixed constant a > such that the LMIs (42) and (43) are met. 



AjY + Y'A/ - BjG - G'B/ + 2-yQj Qj + aY'Af - aG'B/ - Y 
Qj + aAjY-aBjG-Y' -aY-aY' 

Qj>0 



<0 



(42) 
(43) 



with j = 1, ..., r. When the LMIs (42) and (43) are feasible, a state feedback matrix which stabilizes the 
system can be given by (44). 



K= GY- 



(44) 



Proof. Multiplying (42) and (43) by a y > 0, and adding in ;', for ;' = 1 to / = N, LMIs (45) and 
(46) are found. 

( E «jAj)Y + Y'( E kjAj)' - ( E *jBj)G - G'( E «/B ; -)' + 2 7 ( £ a ; Q,) 



/=i 



7=1 

( E «;Q,0 

7 =1 



7=1 



7=1 



/=! 



{L*jAj)Y-a{ZxjBj)G-Y' 

7=1 7=1 



( E oijQj) + aY'( E HjAj)' - aG'( E ocjBj)' - Y 
7=1 7=1 7=1 

-aY-aY 1 

E a j®i > ° 

7=1 



<o 



A{a)Y + Y'A{a)'-B(u)G-G'B(a)'+2yQ(a) Q{a)+aY'A{u)' -aG'B(ot)' -Y 
Q(a) + aA(m)Y - aB(a)G - Y' -aY-aY' 

Q(«) > 



< (45) 
(46) 
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r r 

with Q(a) = j] ttjQj* E a j = 1/ OL; > and ; = l...r. D 

Thus, the uncertain system shown can be fed back in (19) with (45) and (46) sufficient 
conditions for asymptotic stability of the polytope. 

Observation 1. In the LMIs (42) and (43), the constant "a" has to be fixed for all vertexes and to 
satisfy the LMIs and it can be found through a one-dimensional search. 

5.3 Optimization of the K matrix norm using Finsler's lemma 

The motivation for the study of an alternative optimization of the K matrix norm of state 
feedback control was due to less conservative results obtained with Finsler's lemma. This 
way expecting to find, for some situations, controllers with lower gains, thus being easier to 
implement than those designed using the existing quadratic stability theory (Faria et al., 2010), 
avoiding the signal control saturations. 

Some difficulty in applying the existing theorem (Faria et al., 2010) was found to the new 
structure of LMIs, as the controller synthesis matrix Y is not symmetric, a condition that was 
necessary for the development of Theorem (4.1) when the controller synthesis matrix was 
X = P . Thus, Theorem 5.3 is proposed. 

Theorem 5.3. A constraint for the K g ]R mx ™ matrix norm of state feedback can be obtained, with 
K = GY- 1 and Qj = Y'PfY, being Y e R" xn , G € R mxn and P e R" x ", Pj = P- > finding the 
minimum /3, /3 > 0, such that K'K < fiP;, j = 1...N. You can get the optimal value of f> solving the 
optimization problem with the LMIs (47) and (48). 



s.t. 



>0 



'Qj G' 

_ G f,I„, 

■ AjY + Y'A/ - BjG - G'B/ + 2yQj Qj + aY'A/ - aG'B/ 
Qj + aAjY-aBjG-Y' -aY-aY' 

where I m denotes the identity matrix ofm order. 



(47) 
< (48) 



Proof. Applying the Schur complement for (47) results in (49). 

BI m > and Qj - G'{BI m )- l G > (49) 

Thus, from (49), (50) is found. 

Qj>^G'G^G'G< &Qj (50) 

Replacing G = KY and Q ; = Y'PjY in (50), (51) is met. 

Y'K'KY < BY'PjY => K'K < BPj (51) 

on which K is the optimal controller associated with (42) and (43). □ 

Thus it was possible the adequacy of the proposed optimization method with the 
minimization of a scalar &, using the inequality of minimization K'K < BP. with P. the 
Lyapunov's matrix, to the new relaxed parameters. 
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5.4 Stability of systems using reciprocal projection lemma restricted by the decay rate 

Another tool that can be used for stability analysis using LMIs is the reciprocal projection 
lemma (Apkarian et al., 2001) set out in Lemma 5.3. 

Lemma 5.3 (reciprocal projection lemma). Consider Y = Y' > a given matrix. The following 
statements are equivalent 

1. xp + S + S' < 

2. The following LMI is feasible for W 

> + Y- {W + W) S' + W 
S + W -Y 

Proof. Reciprocal projection lemma proof can be found at (Apkarian et al., 2001). □ 



Consider the Lyapunov's inequality subject to a decay rate given by (15) and (16), which can 
be rewritten as (52) and (53). 



(A - BK)X + X{A - BK)' + 2jX < 
X>0 



(52) 
(53) 



where X = P e P is the Lyapunov's matrix. The original Lyapunov's inequality (15) can be 
recovered by multiplying the inequality (52) on the left and on the right by P. 

Assuming if = OeS' = (A — BK)X + yX, it will be verified that the first claim of the reciprocal 
projection lemma will be exactly Lyapunov's inequality subject to the decay rate described in 
(52): 

ip + S + S' = {A- BK)X + X{A - BK)' + 2yX < 

From the reciprocal projection lemma, if the first statement is true, then the second one will 
also be true as (54) shows. 



Y-(W + W) {A - BK)X + yX + W 

X{A - BK)' + -yX + W -Y 



< 



(54) 



Multiplying (54) on the left and on the right by 



/ 

o x- 1 



with P = X- 1 results in (55). 



Y-{W+W) {A-BK) + yI + W'P 
{A-BK)' + JI + PW -PYP 



W l 



/ 



and 



Multiplying (55) on the left and on the right by 
V = W' 1 , (56) is found. 

V'YV-{V+V) V'{A-BK) + yV + P 
(A-BK)'V + jV + P -PYP 



<0 

w- 1 

I 



(55) 



< 



respectively with 



(56) 
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Applying the Schur complement in V'YV, (57) is found. 

-(V+V) V(A-BK) + yV + P V 

(A-BK)'V + yV + P -PYP 

V -Y- 1 

performing the linearizing variable change Y = P results in (58). 

-(V+V) V(A-BK)+yV + P V 

(A-BK)'V + yV + P -P 

V -P 



<0 



(57) 



<0 



(58) 



In literature it can be found a formulation close to the insertion of the decay rate but with 
different positioning of the parameter of decay rate (Shen et al., 2006). It is easy to verify that 
some conservatism was introduced with the choice of Y — P , but the state feedback matrix 
is unrelated to the Lyapunov's matrix P, which results in relaxation of Lyapunov's LMI. Using 
the dual form (A - BK) -> (A - BK)' (Apkarian et al., 2001) results in inequality (59). 



-(V + V) V(A-BK)' + yV + P V 

(A-BK)V + yV + P -P 

V -P. 



<0 



(59) 



Performing the change of variable Z = KV and inserting the constraint P > 0, the LMIs (60) 
and (60) that guarantee system stability can be found. 



-(V + V) V'A'-Z'B' + yV + P V 

AV-BZ + yV + P -P 

V -P 

P > 



< 



(60) 



(61) 



The inequalities (60) and (61) are LMIs, and being feasible, it is deduced a state feedback 
matrix that can stabilize the system (9) - (10) given by (62). 



K = ZV~ 



(62) 



The result of relaxation of LMIs is interesting in the design of robust controllers, proposed 
below. 



5.5 Robust stability of systems using reciprocal projection lemma restricted by the decay 
rate 

A stability analysis for a robust stability condition can be performed considering the 

continuous time linear system an convex combination of r vertexes of the polytope described 

in (20). As in the extended stability case, the advantage of using the reciprocal projection 

lemma for robust stability analysis is the Lyapunov's function degree of freedom, now defined 

r r 

as P(oi) = £ ^i-P;/ E a ; = 1, Dii > ej = l...r, i.e., it is defined a Lyapunov's function Pi for 



7=1 



/=l 
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each vertex j. As described before Theorem 5.2, the use of P(ot) fits to time-invariant polytopic 
uncertainties, being permitted rate of variation sufficiently small. To verify this, Theorem 5.4 
is proposed. 

Theorem 5.4. A sufficient condition which guarantees the stability of the uncertain system (20) is the 
existence of matrices V G R" x ", Pj = P/ G R" xn and Z G R mx ", such that LMIs (63) and (64) are 
met. 



-(V + V) V'A'j-Z'B'j + yV + Pj V 



AjV-BjZ + yV + Pj 
V 



-Pi 



< 



Pj>0 



(63) 
(64) 



with j = 1, ...,r. 



When the LMIs (63) and (64) are feasible, a state feedback matrix which stabilizes the system can be 
given by (65). 

K = ZV- 1 (65) 



Proof. Multiplying (63) and (64) by a ; > 0, and adding in j, for ;' = 1 to ;' = N, (65) and (66) 
are found. 

-(V+V) 

( t *jAj)V- ( t ccjB^Z + 7V+(t ajPj) 

V 

V'( t KjA'j) - Z'( t «jB'j) + 7V' + (L aft) 
/=1 /=1 /=1 



-(LKjPj) 

;=i 



V 
o 



-(E*/Py) 

7=1 



<0 



C£xjPj)>o 

y=i 



-(V+V) V'A'{a) - Z'B'{oc) + jV + P(a) V 

A{ct)V-B{ix)Z + ^V + P(a) -P(a) <0 (65) 

V -P(cc) 

P(a) > (66) 



with P(ol) = E otjPj, E «/ = 1/ «;' > and ;' = l...r. D 

7=1 7=1 
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It appears that K is unique and there are r Lyapunov's matrices Pi, generating a relaxation in 
the LMIs. The same trend was observed in the formulation via Finsler's lemma in which 
variables were the Lyapunov's matrices Qs, but in (65) and (66) there is a greater degree 
of freedom with the inclusion of V in the design of the control matrix K, V being totally 
disconnected from P:, j = \,...,n. 

5.6 Optimization of the K matrix norm using reciprocal projection lemma 

A study was carried out to fit the LMIs to the new relaxed parameters once the state feedback 
matrix K is completely detached from the Lyapunov's matrix P(«). Therefore, relevant 
changes took place in the optimization proposed in this study to suit the reciprocal projection 
lemma. This optimization has provided interesting results in practice. 

Due to the lack of relations to assemble LMI able to optimize the module of K it was proposed 
a minimization procedure similar to the optimization procedure for redesign presented in 
(Chang et al, 2002) inserting an extra restriction to the LMIs (63) and (64). 

Thus Theorem 5.5 was proposed. 

Theorem 5.5. A constraint for the K g ]RJnx« ma i r i x nor m of state feedback is obtained, with 
K = ZV- 1 , V e E" x " and Z e R mx " finding the minimum $, $ > 0, such that K'K < f>M, 
being M = V'^ 1 V^ 1 and therefore M = M' > 0. You can get the optimal value of [5 solving the 
optimization problem with the LMIs (67) and (68). 



min/3 

In Z' 

Z pi m 



s.t 



(67) 
> 



(Set of LMIs (63) and (64)) (68) 

which I m and I„ denote the identity matrices of m and n order respectively. 

Proof. Applying the Schur complement in (67) results in (69). 

f,I m > e I n - Z\f,I m )- l Z > (69) 

Thus, from (69), (70) is found. 

I n > iz'Z => Z'Z < (SJ„ (70) 

Replacing Z = KV in (70) results in (71). 

V'K'KV < f,I„ (71) 

Multiplying on the left and on the right (71) for V' -1 e V -1 respectively and naming 
V'- l V- 1 = M (72) is met. 

V'K'KV < f>I n => K'K < fiM (72) 

where K is the optimal controller associated with (63) and (64). □ 

Due to M being defined as M = V'~ l V~ x and so M = M' > 0, it is possible to find a 
relationship that optimizes the matrix K minimizing a scalar /3, with the relation of minimizing 
K'K < fiM. 
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6. Practical application in the 3-DOF helicopter 

Consider the schematic model in Figure (2) of the 3-DOF helicopter (Quanser, 2002) shown 
in Figure (1). Two DC motors are mounted at the two ends of a rectangular frame and drive 
two propellers. The motors axis are parallel and the thrust vector is normal to the frame. The 
helicopter frame is suspended from the instrumented joint mounted at the end of a long arm 
and is free to pitch about its center (Quanser, 2002). 

The arm is gimbaled on a 2-DOF instrumented joint and is free to pitch and yaw. The other 
end of the arm carries a counterweight such that the effective mass of the helicopter is light 
enough for it to be lifted using the thrust from the motors. A positive voltage applied to the 
front motor causes a positive pitch while a positive voltage applied to the back motor causes 
a negative pitch (angle pitch (p)). A positive voltage to either motor also causes an elevation 
of the body (i.e., pitch of the arm). If the body pitches, the thrust vectors result in a travel of 
the body (i.e., yaw (e) of the arm) as well. If the body pitches, the impulsion vector results in 
the displacement of the system (i.e., travel (A) of the system). 




Fig. 1. Quanser 's 3-DOF helicopter of UNESP - Campus Ilha Solteira. 

The objective of this experiment is to design a control system to track and regulate the 
elevation and travel of the 3-DOF Helicopter. 
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The 3-DOF Helicopter can also be fitted with an active mass disturbance system that will not 
be used in this work. 



Pitch axis 



Back motor 



Travel 



Front motor 




Fig. 2. Schematic drawing of 3-DOF Helicopter 

The state space model that describes the helicopter is (Quanser, 2002) shown in (73). 
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(73) 
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The variables £ and 7 represent the integrals of the angles £ of yaw and A of travel, respectively. 
The matrices A and B are presented in sequence. 



10 0' 
00 10 00 
000100 
0000 00 
000000 



— y.-"Y^ 2 

2mfl/+2m f l h 2 +m-J, 



000000 
100000 



and B 















l.k f 


UCf 


m w T*,+2m f ll 

1 */ 

2 m f l,, 




m-Ji+lnifli 

1 k f 

ImJTi, 

















The values used in the project were those that appear in the MATLAB programs for 
implementing the original design manufacturer, to maintain fidelity to the parameters. The 
constants used are described in Table (1). 



Power constant of the propeller (found experimentally) 


k f 


0.1188 


Mass of the helicopter body (kg) 


m h 


1.15 


Mass of counterweight (kg) 


m w 


1.87 


Mass of the whole front of the propeller (kg) 


rtif 


m h /2 


Mass of the whole back of the propeller (kg) 


m b 


m h /2 


Distance between each axis of pitch and motor (m) 


h 


0.1778 


Distance between the lift axis and the body of the helicopter (m) 


h 


0.6604 


Distance between the axis of elevation and the counterweight (m) 


lw 


0.4699 


Gravitational constant (m/s ) 


g 


9.81 



Table 1. Helicopter parameters 

Practical implementations of the controllers were carried out in order to view the controller 
acting in real physical systems subject to failures. 

The trajectory of the helicopter was divided into three stages. The first stage is to elevate the 
helicopter 27.5° reaching the yaw angle £ = 0°. In the second stage the helicopter travels 
120°, keeping the same elevation i.e., the helicopter reaches A = 120° with reference to the 
launch point. In the third stage the helicopter performs the landing recovering the initial 
angle £ = -27.5°. 

During the landing stage, more precisely in the instant 22 s, the helicopter loses 30% of the 
power back motor. The robust controller should maintain the stability of the helicopter and 
have small oscilation in the occurrence of this failure. 

To add robustness to the system without any physical change, a 30% drop in power of the 
back motor is forced by inserting a timer switch connected to an amplifier with a gain of 0.7 
in tension acting directly on engine, and thus being constituted a polytope of two vertexes 
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with an uncertainty in the input matrix of the system acting on the helicopter voltage between 
0, 7Vj, and Vj,- The polytope described as follows. 

Vertex 1 (100% of V b ): 



10 0' 
00 100 
000 10 
00000 
00000 

-1.2304 00000 

1 00000 
10000 


































and B 






1 














0.0858 


0.0858 


0.5810 


-0.5810 





















Vertex 2 (70% of V b ): 



A 2 



10 0" 

001000 

000 10 

000000 

000000 

-1.230400000 

1 00000 
10000 



and B2 





















0.0858 


0.0601 


0.5810 


-0.4067 





















Fixing the decay rate equal to 0.8, there were designed: a controller with quadratic stability 
using the existing optimization (Assuncao et al., 2007c), a controller with quadratic stability 
with the proposed optimization and controllers with extended stability and projective stability 
also with the proposed optimization to perform the practical implementation. 

The controller designed by quadratic stability with existing optimization (Theorem 4.1) is 
shown in (74) (Assuncao et al., 2007c). 



-46.4092 -15.6262 21.3173 -24.7541 -3.9269 23.5800 -27.4973 7.4713 
-70.3091 13.3795 -10.1982 -37.5960 4.3357 -15.1521 -41.5328 -2.7935 



(74) 



where ||K|| = 107.83. 

This controller was implemented in helicopter and the results are shown in Figure 3. 

In (75) follows the quadratic stability controller design with the proposed optimization follows 
(Theorem 5.1). 

v _ [ -18.8245 -12.2370 10.9243 -13.9612 -4.4480 14.6213 -9.1334 3.2483 1 mn 

*■ ~~ [-27.9219 10.6586 -7.6096-20.1096 4.5602 -11.0774 -13.7202 -2.2629 J V°l 

where | IK 1 1 =44.88. 

This controller was implemented in the helicopter and the results are shown in Figure 4. 

In (76) follows the extended stability controller design with the proposed optimization follows 
(Theorem 5.3). For this LMIs an a = 10 solves the problem. Though the Theorem 5.3 
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and Theorem 5.5 hypothesis establishes a sufficiently low time variation of a. However, for 
comparison purposes of Theorem 5.3 and Theorem 5.5 with Theorem 5.1, the same abrupt loss 
of power test was done with controllers (76) and (77). 

v _ \ -23.7152 -12.9483 9.8587 -18.7322 -4.9737 14.3283 -10.7730 2.6780 1 n ,\ 

^~ [-33.8862 15.2923 -11.6132-25.4922 6.0776 -16.5503 -15.8350 -3.4475 J K - / °> 

where ||K|| =56.47. 

This controller was implemented in the helicopter and the results are shown in Figure (5). 

In (77) follows the projective stability controller design with the proposed optimization 
follows (Theorem 5.5). 



v _ [ -50.7121 -28.7596 35.1829 -29.8247 -7.9563 41.0906 -28.8974 11.7405 1 
*■ ~~ [-66.5405 31.9853 -34.7642-38.3173 9.9376 -42.0298 -38.3418 -11.8207 J 

where 1 1 JC 1 1 = 110.46. 

This controller was implemented in the helicopter and the results are shown in Figure 6. 



(77) 




Fig. 3. Practical implementation of the designed K by quadratic stability with the 
optimization method presented in (Assuncao et al., 2007c). 

The graphics of Figures 3, 4, 5 and 6, refer to the actual data of the angles and voltages on the 
front motor (Vc) and back motor (Vj,) measured with the designed controllers acting on the 
plant during the trajectory described as a failure in the instant 22 s. Tensions (Vc) and (Vj,) on 
the motors were multiplied by 10 to match the scales of the two graphics. 

Note that the variations of the amplitudes of (Vc) and (Vj,) using optimized controllers 
proposed (75) and (76) in Figures 4 and 5 are smaller than those obtained with the existing 
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Fig. 4. Practical implementation of the K designed by quadratic stability with the proposed 
optimization method. 
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Fig. 5. Practical implementation of the K designed by extended stability with the proposed 
optimization method. 



New Techniques for Optimizing the Norm 

of Robust Controllers of Polytopic Uncertain Linear Systems 



95 




Fig. 6. Practical implementation of the K designed by projective stability with the proposed 
optimization method. 

controller in the literature (74) shown in Figure 3. This is due to the fact that our proposed 
controllers (75) and (76) have lower gains then (74). For this implementation the projective 
stability designed controllers with proposed optimization (77) obtained the worst results as 
Figure 6. 

It was checked that the 7 used in the implementation of robust controllers, if higher, forces the 
system to have a quick and efficient recovery, with small fluctuations. 



7. General comparison of the two optimization methods 

In order to obtain more satisfactory results on which would be the best way to optimize the 
norm of K, a more general comparison has been made between the two methods as Theorems 
4.1 and 5.1. 

There were randomly generated 1000 uncertain polytopes of second order systems, with only 
one uncertain parameter (two vertexes) and after that, 1000 uncertain polytopes of fourth 
order uncertain systems, with two uncertain parameter (four vertexes). The 1000 uncertain 
polytopes were generated feasible in at least one case of optimization for 7 = 0.5, and the 
consequences of 7 increase were analyzed and plotted in a bar charts showing the number 
of controllers with lower norm due to the increase of 7, shown in Figure 7 for second-order 
systems and in Figure 8 to fourth-order systems. 

The controllers designed with elevated values of 7 do not have much practical application 
due to the fact that the increase of 7 affect the increasing of the norm and make higher peaks 
of the transient oscillation, used here only for the purpose of analyzing feasibility and better 
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Fig. 7. Number of controllers with lower norm for 1000 uncertain politopic systems of 
second-order randomly generated. 

results for the norm of K, so comparisons were closed in 7 = 100.5, because this 7 is already 
considered high. 

In Figure 8 can be seen that the proposed optimization method produces better results for 
all cases analyzed. Due to the complexity of the poly topes used in this case (fourth-order 
uncertain systems with two uncertainties (four vertexes)), is natural a loss of feasibility with 
the increase of 7, and yet the proposed method shows very good results. 



8. General comparison of the new design and optimization methods 

A generic comparison between the three methods of design and optimization of K was also 
carried out: design by quadratic stability with proposed optimization shown in Theorem 5.1, 
design and proposed optimization with extended stability shown in Theorem 5.3 (using the 
parameter a = 10 in the LMIs) and projective stability design with proposed optimization 
shown in Theorem 5.5. 

Initially 1000 poly topes of second order uncertain systems were randomly generated, with 
only one uncertain parameter (two vertexes) and after that, fourth order uncertain systems, 
with two uncertain parameter (four vertexes). The 1000 polytopes were generated feasible in 
at least one case of optimization for 7 = 0.5 and the consequences of 7 increase were analyzed. 
In fourth-order uncertain systems, the 1000 polytopes were generated feasible in at least one 
case of optimization for 7 = 0.2 and then, the consequences of 7 of 0.2 in 0.2 increase were 
analyzed. This comparison was carried out with the intention of examining feasibility and 
better results for the norm of K. So, a bar graphics showing the number of controllers with 
lower norm with the increase of 7 was plotted, and is shown in Figures 9 and 10. 
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Fig. 8. Number of controllers with lower norm for 1000 uncertain politopic systems of 
fourth-order randomly generated. 
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Fig. 9. Number of controllers with lower norm for 1000 uncertain politopic systems of 
second-order randomly generated. All these methods are proposed in this work. 
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Fig. 10. Number of controllers with lower norm for 1000 uncertain polytopic systems of 
fourth-order randomly generated. All these methods are proposed in this work. 

Both figures 9 and 10 show that the proposed optimization method using quadratic stability 
showed better results for the controller norm with the increase of 7, due to optimization this 
method no longer depend on the matrices that guarantee system stability as it can be seen 
in equation (22). In contrast, using the proposed optimizations with extended stability and 
projective stability, they still depend on the matrices that guarantee system stability as seen in 
equations (51) and (72) and this is the obstacle to finding better results for these methods. 



9. Conclusions 

At the 3-DOF helicopter practical application, the controllers designed with the proposed 
optimization showed lower values of the controller's norm designed by the existing 
optimization with quadratic stability, except the design for projective stability which had the 
worst value of the norm for this case, thus showing the advantage of the proposed method 
regarding implementation cost and required effort on the motors. These characteristics of 
optimality and robustness make our design methodology attractive from the standpoint of 
practical applications for systems subject to structural failure, guaranteeing robust stability 
and small oscillations in the occurrence of faults. 

It is clear that the design of K via the optimization proposed here achieved better results than 
the existing optimizing K (Assuncjao et al., 2007c), using the LMI quadratic stability for second 
order polytopes with one uncertainty. The proposed optimization project continued to show 
better results even when the existing optimization has become totally infeasible for fourth 
order polytopes with two uncertainties. 
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By comparing the three optimal design methods proposed here (quadratic stability, extended 
stability, and projective stability) it can be concluded that the design using quadratic stability 
had a better performance for both analysis: 1000 second order polytopes with one uncertainty 
and for the 1000 fourth order polytopes with two uncertainties, showing so that the proposed 
optimization ensures best results when used with the quadratic stability. 
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1. Introduction 

In last years has been a growing interest of researchers on theory and applications of switched 
control systems, widely used in the area of power electronics (Cardim et al., 2009), (Deaecto 
et al, 2010), (Yoshimura et al, 2011), (Batlle et al, 1996), (Mazumder et al, 2002), (He et al, 
2010) and (Cardim et al., 2011). The switched systems are characterized by having a switching 
rule which selects, at each instant of time, a dynamic subsystem among a determined number 
of available subsystems (Liberzon, 2003). In general, the main goal is to design a switching 
strategy of control for the asymptotic stability of a known equilibrium point, with adequate 
assurance of performance (Decarlo et al., 2000), (Sun & Ge, 2005) and (Liberzon & Morse, 
1999). The techniques commonly used to study this class of systems consist of choosing an 
appropriate Lyapunov function, for instance, the quadratic (Feron, 1996), (Ji et al., 2005) and 
(Skafidas et al., 1999). However, in switched affine systems, it is possible that the modes 
do not share a common point of equilibrium. Therefore, sometimes the concept of stability 
should be extended using the ideas contained in (Bolzern & Spinelli, 2004) and (Xu et al., 
2008). Problems involving stability analysis can many times be reduced to problems described 
by Linear Matrix Inequalities, also known as LMIs (Boyd et al., 1994) that, when feasible, are 
easily solved by some tools available in the literature of convex programming (Gahinet et al., 
1995) and (Peaucelle et al., 2002). The LMIs have been increasingly used to solve various types 
of control problems (Faria et al., 2009), (Teixeira et al., 2003) and (Teixeira et al., 2006). This 
paper is structured as follows: first, a review of previous results in the literature for stability 
of switched affine systems with applications in power electronics is described (Deaecto et al., 
2010). Next, the main goal of this paper is presented: a new theorem, which conditions hold 
when the conditions of the two theorems proposed in (Deaecto et al., 2010) hold. Later, in 
order to obtain a design procedure more general than those available in the literature (Deaecto 
et al., 2010), it was considered a new performance indice for this control system: bounds on 
output peak in the project based on LMIs. The theory developed in this paper is applied to 
DC-DC converters: Buck, Boost, Buck-Boost and Sepic. It is also the first time that this class 
of controller is used for controlling a Sepic DC-DC converter. The notation used is described 
below. For real matrices or vectors (') indicates transpose. The set composed by the first N 
positive integers, 1, ..., N is denoted by K. The set of all vectors A = (Aj, . . . , A^)' such that 
A, > 0, i = 1, 2, . . . , N and Aj + A2 + . . . + A^ = 1 is denoted by A. The convex combination 
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N 

of a set of matrices {A\, . . . , A-^) is denoted by A\ = V"' A,A,-, where AG A. The trace of a 

1=1 
matrix P is denoted by Tr(P). 

2. Switched affine systems 

Consider the switched affine system defined by the following state space realization: 

* = K(t) x + B a(t) w , *(0) = x (1) 

y = c a(t )X, (2) 

as presented in (Deaecto et al., 2010), were x(t) G R" is the state vector, y(t) G IR P is the 
controlled output, w 6 1R'" is the input supposed to be constant for all f > and o~(t): t > 
-*■ K is the switching rule. For a known set of matrices A; G E" x ", B, eE™ and C; G W, 
i = l,...,N, such that: 

•A<r(i) G {A!,A 2 ,...,A N }, (3) 

B (7(t) G{B 1 ,B 2 ,...,B N }, (4) 

Q(f) G { C l' C 2/ • • • / Ov} / (5) 

the switching rule cr(t) selects at each instant of time t > 0, a known subsystem among the 
N subsystems available. The control design problem is to determine a function cr(x(t)), for 
all t > 0, such that the switching rule c(t), makes a known equilibrium point x = x r of (1), 
(2) globally asymptotically stable and the controlled system satisfies a performance index, for 
instance, a guaranteed cost. The paper (Deaecto et al., 2010) proposed two solutions for these 
problems, considering a quadratic Lyapunov function and the guaranteed cost: 



/■CO rOO 

i/ (y - Co-Xr)' (y - CcrX r )dt = min / (x— x r )'Qcr(x - Xr)dt, (6) 

.Jo ueKio 

where Q a = C'^Ca > for all a G K. 



mm 

<7gk Jo 



2.1 Previous results 

Theorem 1. (Deaecto et ah, 2010) Consider the switched affine system (1), (2) with constant input 
w(t) = w for all t > and let the equilibrium point x r G 1R" be given. If there exist A G A and a 
symmetric positive definite matrix P G IR" X " such that 

A' X P + PA X + Q X <Q, (7) 

A x x r + B x w = 0, (8) 
then the switching strategy 

a(x) = ar S mm.g{Q£+2P{AiX + Biw)), (9) 

jgK 

where Q, = C'jCi and £ = x — x r , makes the equilibrium point x r G IR" globally asymptotically stable 
and from (6) the guaranteed cost 

rCO 

J=l (y-CcrXrYiy-Ca-X^dtKiXo-XrYPixo-Xr), (10) 

holds. 
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Proof. See (Deaecto et al., 2010). □ 

Remembering that similar matrices have the same trace, it follows the minimization problem 
(Deaecto et al., 2010): 

inf {Tr{P) :A' A P + PA A + Q A < 0, AG A}. (11) 

The next theorem provides another strategy of switching, more conservative, but easier and 
simpler to implement. 

Theorem 2. (Deaecto et al, 2010) Consider the switched affine system (1), (2) with constant input 
w(t) = w for all t > and let the equilibrium point x r g TR" be given. If there exist A £ A, and a 
symmetric positive definite matrix P g JR" X " such that 

A\P + PA, + Qi < 0, (12) 

A A x r + B A w = 0, (13) 
for all i G K, then the switching strategy 

a(x) = arg mm. ^'P(AiX r + Bjw), (14) 

where £ = x — x r , makes the equilibrium point x r 6 JR" globally asymptotically stable and the 
guaranteed cost (10) holds. 

Proof. See (Deaecto et al., 2010). □ 

Theorem 2 gives us the following minimization problem (Deaecto et al., 2010): 

inf { Tr(P) : A\P + PA { + Q,- < 0, i G K} . (15) 

Note that (12) is more restrictive than (7), because it must be satisfied for all i G K. However, 
the switching strategy (14) proposed in Theorem 2 is simpler to implement than the strategy 
(9) proposed in Theorem 1, because it uses only the product of f by constant vectors. 

2.2 Main results 

The new theorem, proposed in this paper, is presented below. 

Theorem 3. Consider the switched affine system (1), (2) with constant input w(t) = w for all t > 
and let x r G 1R" be given. If there exist A G A, symmetric matrices N,-, i G K and a symmetric positive 
definite matrix P G JR" X " such that 

A[P + PA i + Q i -N i <0, (16) 

A k x r + B X W = 0, (17) 

N A = 0, (18) 

for all i G K, where Q,- = Q',, then the switching strategy 

c-(x) = argmmg'(Ni£+ 2P(A t x r + Bjw)), (19) 

where £ = x — x r , makes the equilibrium point x r G IR" globally asymptotically stable and from (10), 
the guaranteed cost J < [xq — x t )'P(xq — x r ) holds. 
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Proof. Adopting the quadratic Lyapunov candidate function V(f) = g'-Pg and from (1), (16), 
(17) and (18) note that for g ^ 0: 

V(g) = x'Pg + g'Pi = 2Z'P(A a x + B^w) = £(A' a P + PA a )l + 2£'P(A a x r + B a w) 

< ?{-Qa + N a )£ + 2gP{A a x r + B a w) = f (N^g + 2P(A a x r + B a w)) - g'Q^g 
= min {?(Ni£ + 2P(A,x r + B.ia))} - g'Q^g 

= m&n{?(N K Z + 2P(A x Xr + B K w))} -fQaS 

< -g'Q^g < 0. (20) 

Since V(£) < for all £ ^ G R", and V (0) = 0, then x r G R" is an equilibrium point globally 
asymptotically stable. Now, integrating (20) from zero to infinity and taking into account that 
V(g(oo)) = 0, we obtain (10). The proof is concluded. □ 

Theorem 3 gives us the following minimization problem: 

inf { Tr{P) : A[P + PA t + Q, - N, < 0, N A = 0, i G K} . (21) 

The next theorem compares the conditions of Theorems 1, 2 and 3. 

Theorem 4. The following statements hold: 

(i) if the conditions of Theorem 1 are feasible, then the conditions of Theorem 3 are also feasible; 

(ii) if the conditions of Theorem 2 are feasible, then the conditions of Theorem 3 are also feasible. 

Proof. (/') Consider the symmetric matrices N,, i G K, as described below: 

N { = (A'iP + PA, + Q,)-(A' A P + PA A + Q A ) . (22) 

Then, multiplying (22) by A,- and taking the sum from 1 to N it follows that 

N N N 

N A = £ A,N, = £ MA'.P + PA, + Qi) - £ A;(^ A P + PA A + Q A ) 
i=l i=i 1=1 

= (A' X P + PA X + Q A ) - (A' X P + PA X + Q A ) = 0. (23) 

Now, from (16), (18) and (22) observe that 

A\P + PA, + Q, - N, = A^P + PA, + Qt - [[A',P + PA, + Q,) - (A' A P + PA A + Q A )) 

= A' A P + PA A + Q A < 0, Vi G K. (24) 
(ii) It follows considering that N; = in (16): 

A\P + PA, + Q, - N, = A^ + PAj + Q, < 0, Vi G K. (25) 

Thus, the proof of Theorem 4 is completed. □ 
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2.3 Bounds on output peak 

Considering the limitations imposed by practical applications of control systems, often must 
be considered constraints in the design. Consider the signal: 

s = HS, (26) 

where H £ R' x " is a known constant matrix, and the following constraint: 

max||s(f)ll < tpo, (27) 

(>0 



where ||s(f)|| = -y/s(f)'s(f) and lp is a known positive constant, for a given initial condition 
f (0). In (Boyd et al., 1994), for an arbitrary control law were presented two LMIs for the 
specification of these restrictions, supposing that there exists a quadractic Lyapunov function 
V(£) = f P£, with negative derivative defined for all f ^ 0. For the particular case, where 
s(f) = y(t), with y(t) G JR P defined in (2), is proposed the following lemma: 

Lemma 1. For a groen constant tp > 0, if there exist A e A, and a symmetric positive definite matrix 
P G R" x ", solution of the following optimization problem, for all i G K: 



P C\ 



> 0, (28) 



*» f(0)'P 
Pf(0) P 



> 0, (29) 



(Set of LMIs), (30) 

a>/zere (Sef o/ LMIs) can be equal to (7)-(8), (12)-(13) or (16)-(18) then the equilibrium point £ = 
x — x r = is globally asymptotically stable, the guaranteed cost (10) and f/ze constraint (27) r;o/ii. 

Proof. It follows from Theorems 1, 2 and the condition for bounds on output peak given in 
(Boyd et al., 1994). □ 

The next section presents applications of Theorem 3 in the control design of three DC-DC 
converters: Buck, Boost and Buck-Boost. 

3. DC-DC converters 

Consider that ii(t) denotes the inductor current and V c (t) the capacitor voltage, that were 
adopted as state variables of the system: 

x(t) = [x 1 (t)x 2 (t)]' = [i L (t)V c (t)}'. (31) 

Define the following operating point x r = [x lr Xjr]' = [»'j> V cr ]'. Consider the DC-DC power 
converters: Buck, Boost and Buck-Boost, illustrated in Figures 1, 3 and 5, respectively. The 
DC-DC converters operate in continuous conduction mode. For theoretical analysis of DC-DC 
converters, no limit is imposed on the switching frequency because the trajectory of the system 
evolves on a sliding surface with infinite frequency. Simulation results are presented below. 
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The used solver was the LMILab from the software MATLAB interfaced by YALMIP (Lofberg, 
2004) (Yet Another LMI Parser). Consider the following design parameters (Deaecto et al., 
2010): V g = 100[V], R = 50[fl], r L = 2[n], L = 5Q0[}iH], C = 470[^F] and 

0=0 = \ plTL ° 

is the performance index matrix associated with the guaranteed cost: 

J o (p 2 R~\V c - Vcrf + Pl r L (i L - l Lr ) 2 dt, 

where p\ and p 2 G IR+ are design parameters. Note that p, G 1R + plays an important role with 
regard to the value of peak current and duration of the transient voltage. Adopt p\ = and 
Pi = 1- 

3.1 Buck converter 




Fig. 1. Buck DC-DC converter. 

Figure 1 shows the structure of the Buck converter, which allows only output voltage of 
magnitude smaller than the input voltage. The converter is modeled with a parasitic resistor 
in series with the inductor. The switched system state-space (1) is defined by the following 
matrices (Deaecto et al., 2010): 



-r L /L -1/L 
1/C -1/RC 



A 2 



-r L /L -1/L 
1/C -1/RC 



1/L 




B 2 



(32) 



In this example, adopt Aj = 0.52 and \ 2 = 0.48. Using the minimization problems (11) 
and (15), corresponding to Theorems 1 and 2, respectively, we obtain the following matrix 
quadratic Lyapunov function 



1 x 10 



-4 



0.0253 0.0476 
0.0476 0.1142 
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needed for the implementation of the switching strategies (9) and (14). Maintaining the same 
parameters, from minimization problem of Theorem 3, we found the matrices below as a 
solution, and from (10) the guaranteed cost / < (xg — x r )'P{xQ — x r ) = 0.029: 



P = 1 x 10~ 4 



0.0253 0.0476 
0.0476 0.1142 



N-, 



-1 x 10" 



0.2134 0.0693 
0.0693 0.0685 



N, = lx 10" 



0.2312 0.0751 
0.0751 0.0742 



The results are illustrated in Figure 2. The initial condition was the origin x = [zjr_ V c ] = [0 0]' 
and the equilibrium point is equal to x r = [1 50]'. 




10 15 20 25 30 35 

(a) Phase plane. 



0.2 0.4 0.6 



I 1 1.2 1.4 1.6 

Time (s) 



(b) Normalized Lyapunov functions 



V(x(0))- 




TheoA Theo. 2 Tlieo. 3 



0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.01 
Time (s) 

(c) Voltage. 




0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.01 
Tune (s) 

(d) Current. 



Fig. 2. Buck dynamic. 

Observe that Theorem 3 presented the same convergence rate and cost by applying Theorems 
1 and 2. This effect is due to the fact that for this particular converter, the gradient of the 
switching surface does not depend on the equilibrium point (Deaecto et al., 2010). Table 1 
presents the obtained results. 



3.2 Boost converter 

In order to compare the results from the previous theorems, designs and simulations will be 
also done for a DC-DC converter, Boost. The converter is modeled with a parasitic resistor 
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Table 1. Buck results. 





Overshoot [A] 


Time [ms] 


Cost (6) 


Theo. 1 


36.5 


2 


0.029 


Theo. 2 


36.5 


2 


0.029 


Theo. 3 


36.5 


2 


0.029 



Ril^-4- 




Fig. 3. Boost DC-DC converter. 

in series with the inductor. The switched system state-space (1) is defined by the following 
matrices (Deaecto et al., 2010): 



-r L /L 
-1/RC 



A 2 



-rJL -1/L 
1/C -1/RC 



"1/L" 


, B 2 = 


"1/L" 











(33) 



In this example, Aj = 0.4 and A 2 = 0.6. Using the minimization problems (11) of Theorem 1 
and (15) of Theorem 2, the matrices of the quadratic Lyapunov functions are 



p = 1 x 10~ 4 



0.0237 0.0742 
0.0742 0.2573 



lx 10" 



0.1450 0.0088 
0.0088 0.2478 



respectively. Now, from minimization problem of Theorem 3, we found the matrices below as 
a solution, and from (10) the guaranteed cost / < (xg — x r )'P{xQ — x r ) = 0.59: 



1 x 10 



-4 



0.0237 0.0742 
0.0742 0.2573 



Ni 



-0.018 -0.030 
-0.030 0.0178 



N 2 



0.012 0.020 
0.020 -0.012 



The initial condition is the origin and the equilibrium point is x r = [5 150]'. The results are 
illustrated in Figure 4 and Table 2 presents the obtained results. 
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(a) Phase plane. 



(b) Normalized Lyapunov functions 
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Fig. 4. Boost dynamic. 



Table 2. Boost results. 





Overshoot [A] 


Time [ms] 


Cost (6) 


Theo. 1 


36.5 


7 


0.59 


Theo. 2 


36.5 


40 


5.59 


Theo. 3 


36.5 


7 


0.59 



3.3 Buck-Boost converter 

Figure 5 shows the structure of the Buck-Boost converter. The switched system state-space (1) 
is defined by the following matrices (Deaecto et al., 2010): 



-n/L o 

-l/RC 



A 2 



-r L /L -\/L 
\IC -l/RC 



l/L 





B 2 



(34) 



The initial condition was the origin x = [ii V c ]' = [0 0]', Aj = 0.6, A 2 = 0-4 and the 
equilibrium point is equal to x r = [6 120]' . Moreover, the optimal solutions of minimization 
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Fig. 5. Buck-Boost DC-DC converter. 

problems (11) of Theorem 1 and (15) of Theorem 2, are 



P = 1 x 1CT 4 



0.0211 0.0989 
0.0989 0.4898 



1 x 10" 



0.1450 0.0088 
0.0088 0.2478 



respectively. Maintaining the same parameters, the optimal solution of minimization problem 
(21) are the matrices below and from (10) the guaranteed cost / < (xq — x r )'P(xo — x r ) = 0.72: 



1 x 10 



-4 



0.0211 0.0990 
0.0990 0.4898 



Ni 



-0.0168 -0.0400 
-0.0400 0.0158 



N 2 



0.0253 0.0600 
0.0600 -0.0237 



The results are illustrated in Figure 6. Table 3 presents the obtained results. The next section 





Overshoot [A] 


Time [ms] 


Cost (6) 


Theo. 1 


37.5 


10 


0.72 


Theo. 2 


7.5 


70 


3.59 


Theo. 3 


37.5 


10 


0.72 



Table 3. Buck-Boost results. 

is devoted to extend the theoretical results obtained in Theorems 1 (Deaecto et al., 2010) and 2 
(Deaecto et al., 2010) for the model Sepic DC-DC converter. 



4. Sepic DC-DC converter 

A Sepic converter (Single-Ended Primary Inductor Converter) is characterized by being able 
to operate as a step-up or step-down, without suffering from the problem of polarity reversal. 
The Sepic converter consists of an active power switch, a diode, two inductors and two 
capacitors and thus it is a nonlinear fourth order. The converter is modeled with parasitic 
resistances in series with the inductors. The switched system (1) is described by the following 
matrices: 



l/Z-i 













1/L, 







1/Cj 


-1/L 2 








, B 1 = 















-1/(RC 2 )J 
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Fig. 6. Buck-Boost dynamic. 




Fig. 7. Sepic DC-DC converter. 

A 2 



-1/Li -1/Lj 
-r L2 /L 2 1/L 2 

1/Ci 

1/C 2 -1/C 2 -1/(RC 2 ) 



B 2 



1/Li 






(35) 



For this converter, consider that in(t), i 1.2(f) denote the inductors currents and V c \(t), V c2 (t) 
the capacitors voltages, that again were adopted as state variables of the system: 



X(t) = [xi(t) X 2 (t) X 3 (f) X4(t)]' = foiW iu(t) V cl (t) V c2 (t)]>. 



(36) 
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Adopt the following operating point, 

x r = [xi,(f) x 2r (t) x 3r (t) x ir (t)}' = [i Llr (t) i L2r (t) V clr (t) V c2r (t)]' ■ 



(37) 



The DC-DC converter operates in continuous conduction mode. The used solver was the 
LMILab from the software MATLAB interfaced by YALMIP (Lofberg, 2004) . The parameters 
are the following: V g = 100[V], R = 50 [fl], r L1 = 2[fi], r L2 = 3[f>], L x = 500 [>H], L 2 = 
600[}iH], Ci = 800 \jtF], C 2 = 470[}iF} and 



(38) 



Pirn 





p 2 r L2 












Q, = Q 

o 

0p 3 /R_ 
is the performance index matrix associated with the guaranteed cost 

(Pimihl - krl) 2 + PiTLlij-lZ - krl) 2 + P3R^{Vc2 - V c2r ) 2 ) it, 



Jo 



(39) 



where p,- € 1R+ are design parameters. Before of all, the set of all attainable equilibrium point 
is calculated considering that 



{ [klr k2r Vclr W : V clr = Vg, < V c2r < Ri L2r } . 



(40) 



The initial condition was the origin x = [in ijji V c \ V c2 ] = [0 0]'. Figure 8 shows 
the phase plane of the Sepic converter corresponding to the following values of load voltage 
V c2r = {50, 60,..., 150}. 

In this case, Theorem 1 presented a voltage setting time smaller than 30 [ms] and the maximum 
current peak in = 3A[A] and i^ 2 = 9[A}. However, Theorem 2 showed a voltage setting time 
smaller than 80 [ms], with currents peaks in = 34 [A] and ii 2 = 13.5 [A]. Now, in order to 
compare the results from the proposed Theorem 3, adopt origin as initial condition, Aj = 
0.636, A 2 = 0.364 and the equilibrium point equal to x r = [5.24 - 3 100 150]'. From the 
optimal solutions of minimization problems (11) and (15), we obtain respectively 

0.0141 -0.0105 0.0037 0.0707 

-0.0105 0.0078 -0.0026 -0.0533 

0.0037 -0.0026 0.0016 0.0172 

0.0707 -0.0533 0.0172 0.3805 



1 x 10 



-4 



1 x 10 



-3 



0.0960 -0.0882 0.0016 0.0062 

-0.0882 0.0887 0.0184 -0.0034 

0.0016 -0.0184 0.0940 0.0067 

0.0062 -0.0034 0.0067 0.2449 
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Fig. 8. Sepic DC-DC converter phase plane. 

Maintaining the same parameters, the optimal solution of minimization problem (21) are the 
matrices below and from (10) the guaranteed cost / < (xq — x r )'P(xQ — x r ) = 0.93: 

0.0141 -0.0105 0.0037 0.0707 

-0.0105 0.0078 -0.0026 -0.0533 
1 x 10" " 

0.0037 -0.0026 0.0016 0.0172 

0.0707 -0.0533 0.0172 0.3805 



Ni 



-0.0113 0.0099 0.0003 -0.0286 
0.0099 -0.0085 0.0002 0.0290 
0.0003 0.0002 0.0009 0.0088 

-0.0286 0.0290 0.0088 0.0168 



N 2 



0.0197 -0.0173 -0.0005 0.0500 
-0.0173 0.0148 -0.0003 -0.0507 
-0.0005 -0.0003 -0.0015 -0.0154 

0.0500 -0.0507 -0.0154 -0.0293 
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The results are illustrated in Figure 9 and Table 4 presents the obtained results from the 
simulations. 
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Fig. 9. Sepic dynamic. 





Overshoot [A] 


Time [ms] 


Cost (6) 


Theo. 1 


34 


30 


0.93 


Theo. 2 


34 


80 


6.66 


Theo. 3 


34 


30 


0.93 



Table 4. Sepic results. 
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Remark 1. From the simulations results, note that the proposed Theorem 3 presented the same results 
obtained by applying Theorem 1. Theorem 3 is an interesting theoretical result, as described in Theorem 

4. and the authors think that it can be useful in the design of more general switched controllers. 

5. Conclusions 

This paper presented a study about the stability and control design for switched affine 
systems. Theorems proposed in (Deaecto et al., 2010) and later modified to include bounds on 
output peak on the control project were presented. A new theorem for designing switching 
affine control systems, with a flexibility that generalises Theorems 1 and 2 from (Deaecto et al., 
2010) was proposed. Finally, simulations involving four types of converters namely Buck, 
Boost, Buck-Boost and Sepic illustrate the simplicity, quality and usefulness of this design 
methodology. It was also the first time that this class of controller was used for controlling a 
Sepic converter, that is a fourth order system and so is more complicated than the switched 
control design of second order Buck, Boost and Buck-Boost converters (Deaecto et al., 2010). 
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1. Introduction 

Classification of processes and tuning of the PID controllers is initiated by Ziegler and 
Nichols (1942). This methodology, proposed seventy years ago, is still actual and 
inspirational. Process dynamics characterization is defined in both the time and frequency 
domains by the two parameters. In the time domain, these parameters are the velocity gain 
K v and dead-time L of an Integrator Plus Dead-Time (IPDT) model GzN(s)=K v exp(-Ls)/s, 
defined by the reaction curve obtained from the open-loop step response of a process. In the 
frequency domain these parameters are the ultimate gain k u and ultimate frequency o u , 
obtained from oscillations of the process in the loop with the proportional controller k=k u . 
The relationship between parameters in the time and frequency domains is determined by 
Ziegler and Nichols as 

L= * Kv =A, = , ZN =i. (1) 

2m u k u n 

However, for the process G p (s)=Gzn(s) in the loop with the proportional controller k, one 
obtains from the Nyquist stability criterion the same relationship (1) with e-1. As a 
consequence, from (1) and the Ziegler-Nichols frequency response PID controller tuning, 
where the proportional gain is k=0.6k n , one obtains the step response tuning k=03eii/(K v L). 
Thus, for £=£zn one obtains k=1.2/(K v L), as in (Ziegler & Nichols, 1942), while for £=1 one 
obtains fc=0.9425/(K v L), as stated in (Astrom & Hagglund, 1995a). According to (1), the same 
values of the integral time Ti=Ji/o u and derivative time Td=0.257r/ a u are obtained in both 
frequency and time domains, in (Ziegler & Nichols, 1942) and from the Nyquist analysis. 
This will be discussed in more detail in Section 2. 

Tuning formulae proposed by Ziegler and Nichols, were improved in (Hang et al., 1991; 
Astrom & Hagglund, 1995a; 1995b; 2004). Besides the ultimate gain k u and ultimate 
frequency a u of process G p (s), the static gain _K p =G(0), for stable processes, and velocity gain 
K v =lim s ^ sG p (s) , for integrating processes, are used to obtain better process dynamics 

characterization and broader classification (Astrom et al.,1992). Stable processes are 
approximated by the First-Order Plus Dead-Time (FOPDT) model GFo(s)=K p exp(-Ls)/(Ts+l) 
and classified into four categories, by the normalized gain Ki=Kpk u and normalized dead- 
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time 9i=L/T. Integrating processes are approximated by the Integrating First-Order Plus 
Dead-Time (IFOPDT) model Gif(s)=K v exp(-Ls)/ (s(T v s+l)) and classified into two categories, 
by the normalized gain K2=K v k u /a> u and normalized dead-time 02=L/T V . The idea of 
classification proposed in (Astrom et al., 1992) was to predict the achievable closed-loop 
performance and to make possible performance evaluation of feedback loops under closed- 
loop operating conditions. 

In the present chapter a more ambitious idea is presented: define in advance the PID 
controller parameters in a classification plane for the purpose of obtaining a PID controller 
guaranteeing the desired performance/robustness tradeoff for the process classified into the 
desired region of the classification plane. It is based on the recent investigations related to: I) 
the process modeling of a large class of stable processes, processes having oscillatory 
dynamics, integrating and unstable processes, with the ultimate gain k u (Sekara & Matausek, 
2010a; Matausek & Sekara, 2011), and optimizations of the PID controller under constraints 
on the sensitivity to measurement noise, robustness, and closed-loop system damping ratio 
(Sekara & Matausek, 2009,2010a; Matausek & Sekara, 2011), II) the closed-loop estimation of 
model parameters (Matausek & Sekara, 2011; Sekara & Matausek, 2011b, 2011c), and III) the 
process classification and design of a new Gain Scheduling Control (GSC) in the parameter 
plane (Sekara & Matausek, 2011a). 

The motive for this research was the fact that the thermodynamic, hydrodynamic, chemical, 
nuclear, mechanical and electrical processes, in a large number of plants with a large 
number of operating regimes, constitutes practically an infinite batch of transfer functions 
Gp(s), applicable for the process dynamics characterization and PID controller tuning. Since 
all these processes are nonlinear, some GSC must be applied in order to obtain a high 
closed-loop performance/ robustness tradeoff in a large domain of operating regimes. 
A direct solution, mostly applied in industry, is to perform experiments on the plant in 
order to define GSC as the look-up tables relating the controller parameters to the chosen 
operating regimes. The other solution, more elegant and extremely time-consuming, is to 
define nonlinear models used for predicting accurately dynamic characteristics of the 
process in a large domain of operating regimes and to design a continuous GSC (Matausek 
et al., 1996). However, both solutions are dedicated to some plant and to some region of 
operating regimes in the plant. The same applies for the solution defined by a nonlinear 
controller, for example the one based on the neural networks (Matausek et al., 1999). 

A real PID controller is defined by Fig. 1, with C(s) and Cff(s) given by 



C(s) = fcdS . T +fo ^ l F c( s ) > C ff (*) = ^^-FcW • '<« = hk ' f c(s) - 1, < b . 
s(T ( s + 1) s 



(2) 




Fig. 1. Process G p (s) with a two-degree-of-freedom controller. 
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An effective implementation of the control system (2) is defined by relations 

U( S ) = (k(bR( S )-Y c (s)) + ^(R(s)-Y c (s))-k d sY c ( S ))F c ( S ),Y { (s) = ^-, (3) 

V s J I ( s + 1 

for Fc(s)=l as in (Panagopoulos et al., 2002; Matausek & Sekara, 2011). When the 
proportional, integral, and derivative gains (k, k\, k&) and derivative (noise) filter time 
constant Tt are determined, parameter b can be tuned as proposed in (Panagopoulos et al., 
2002). The PID controller (2), Fc(s)=l, can be implemented in the traditional form, when 
noise filtering affects the derivative term only if some conditions are fulfilled (Sekara & 
Matausek, 2009). The derivative filter time constant T{ must be an integral part of the PID 
optimization and tuning procedures (Isaksson & Graebe, 2002; Sekara & Matausek, 2009). 

For Fc(s) given by a second-order filter, one obtains a new implementation of the Modified 
Smith Predictor (Matausek & Micic, 1996, 1999). The MSP-PID controller (3) guarantees 
better performance/ robustness tradeoff than the one obtained by the recently proposed 
Dead-Time Compensators (DTC's), optimized under the same constraints on the sensitivity 
to measurement noise and robustness (Matausek & Ribic, 2012). 

Robustness is defined here by the maximum sensitivity M s and maximum complementary 
sensitivity M p . The sensitivity to measurement noise Mn, M s , and M p are given by 



M 



1 + L(i&>) 



M = max 



L(iw) 



1 + L(ia>) 



M n =max|C nu (i«)|, (4) 



where L(s) is the loop transfer function and C nu (s) is the transfer function from the 
measurement noise to the control signal. In the present chapter, the sensitivity to the high 
frequency measurement noise is used Mn=M n „, where M n „= | C nu (s) | s ^». 

2. Modeling and classification of stable, integrating, and unstable plants 

A generalization of the Ziegler-Nichols process dynamics characterization, proposed by 
Sekara and Matausek (2010a), is defined by the model 

Aw n exp(-rs) 1 cp _ co u KG v {0) 

m ^ s >~~2 — 2 — a ; — :t~ ' — ' i — r ^ /n\ ' \ > 

s A + < - Acu u exp(-rs) k u co u 1 + fc u G p (0) 

where (p is the angle of the tangent to the Nyquist curve Gp(io) at o u and G p (0) is the gain at 
the frequency equal to zero. Thus, for integrating processes G p (0)=+co and A=a u . Adequate 
approximation of G p (s) by the model G m (s) is obtained for a u =a„ where arg{G p (ico^)}=-7i. It is 
demonstrated in (Sekara & Matausek, 2010a; Matausek & Sekara, 2011, Sekara & Matausek, 
2011a) that this extension of the Ziegler-Nichols process dynamics characterization, for a 
large class of stable processes, processes with oscillatory dynamics, integrating and unstable 
processes, guarantees the desired performance/ robustness tradeoff if optimization of the 
PID controller, for the given maximum sensitivity M s and given sensitivity to measurement 
noise Mn, is performed by applying the frequency response of the model (5) instead of the 
exact frequency response G p (io). 
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Ziegler and Nichols used oscillations, defined by the impulse response of the system 

G» = - " £47 , (6) 



to determine k u and co u , and to define tuning formulae for adjusting parameters of the P, PI 
and PID controllers, based on the relationship between the quarter amplitude damping ratio 
and the proportional gain k. Oscillations defined by the impulse response of the system (6) 
are used in (Sekara & Matausek, 2010a) to define model (5), obtained from G m (s)«G p (s) and 
the relation 

fc u G m( s ) = Aa >u exp(-rs) 

Then, by analyzing these oscillations, it is obtained in (Sekara & Matausek, 2010a) that 
amplitude A=co u k/ (1+k), K=fc u G p (0), and dead-time r is defined by a u and a parameter cp, 
given by 



'dG (ico) 



(8) 



Other interpretation of amplitude A- Ao, obtained in (Matausek & Sekara, 2011), is defined 
by 



k„ 



8G p (ia>) 



ceo 



(9) 



Amplitudes A and Ao are not equal, but they are closely related for stable and unstable 
processes, as demonstrated in (Matausek & Sekara, 2011) and Appendix. Parameter Ao is not 
used for integrating processes, since for these processes A=co a . 

The quadruplet {k u , a u , cp, A} is used for classification of stable processes, processes with 
oscillatory dynamics, integrating and unstable processes in the p-cp parameter plane, defined 
by the normalized model (5), given by 

G„( W )= 2 f XP( -^ } ,s n =^, (10) 

where p=A/co u . From the Nyquist criterion it is obtained that the region of stable processes is 
defined by < <p < n / yjp + 1, < p < 1 (Sekara & Matausek, 2011a). Integrating processes, 
since A=o u , are classified as p = 1, < co < n / v2 processes, while unstable processes are 
outside these regions. It is demonstrated that a large test batch of stable and integrating 
processes used in (Astrom & Hagglund, 2004) covers a small region in the p-cp plane. 

To demonstrate that besides k u and o u , parameters cp and G p (0) must by used for the 
classification of processes, Nyquist curves are presented in Fig. 2 for stable, integrating and 
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unstable processes having the same values fc u =l and ffl u =l. For processes having also the 
same values of (p, the distinction of the Nyquist curves in the broader region around the 
critical point requires the information about gain G p (0), as demonstrated in Fig. 2-a. On the 
other hand, the results presented in Fig. 2-b to Fig. 2-d demonstrate that for the same values 
of k w a u , and G p (0) the distinction of the Nyquist curves in the region around the critical 
point is obtained by applying parameter (p. This fact confirms importance of parameter <p in 
process modeling for controller tuning, taking into account that optimization of the PID 
controller under constraints on the robustness is performed in the region around a u . 




Fig. 2. Nyquist curves of processes with the same values k n =l, ffl u =l: a) (p=7r/4, stable G p (0)=l 
(dashed), integrating G p (0)=co (solid), unstable G p (0)=-2 (dashed-dotted); b) stable processes 
with G p (0)=l, for (p=7r/4 (dashed), (p=7i/6 (solid), cp= n/3 (dashed-dotted); c) integrating 
processes with tp=l (dashed), (p= ji/4 (solid), (p=1.2 (dashed-dotted); d) unstable processes 
with G p (0)= -2, for cp= n/A (dashed), cp= n/6 ( solid), cp= n/3 (dashed-dotted). 

For the lag dominated process 



G, (s) = 1 / cosh v2s 



(11) 



and the corresponding models, the step and impulse responses, with the Nyquist curves 
around o u , are presented in Fig. 3. The models are Ziegler-Nichols IPDT model 
G Z N(s)=-K v exp(-Ls) / s and model (5), with A=a u A: u G p (0)/(l+A: u G p (0)) and A=Aq. The set-point 
and load disturbance step responses of this process, in the loop with the optimal PID 
controller (Matausek & Sekara, 2011) and PID controller tuned as proposed by Ziegler and 
Nichols (1942), are compared in Fig. 4-a. In this case fc u =11.5919, a u =9.8696 and K v =0.9251, 
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L=0.1534. The PID controller tuned as proposed by Ziegler and Nichols is implemented in 
the form 



U{s) = k(bR(s) - Y(s)) + ^-{R(s) - Y(s)) - 



k d s 
T f s + 1 



Y(s) , b = 0,k l 



—,k d =kT d ,T £ =^- 



(12) 



where k=0.6k w Ti=Ji/o n , Td= 7r/(4ff> u ), for the frequency domain ZN tuning (ZN PID1). For 
the time domain ZN tuning (ZN PID2) the parameters are k=1.2/(K v L), Ti=2L, Td=L/2, or, as 
suggested by the earlier mentioned Nyquist analysis, proportional gain k is adjusted to 
k=0.9A3/(K v L), denoted as the modified time domain ZN tuning (ZN ModifPID2). In 
M n = (N d +1) \k\ parameter Nd is adjusted to obtain the same value of M n = 76.37 used in the 
constrained optimization of the PID in (3), Fc(s)=l, where M n = | kd | /Tf. 

Parameters of the PID controllers and performance/ robustness tradeoff are compared in 
Table 1. It is impressive that Ziegler and Nichols succeeded in defining seventy years ago 
an excellent experimental tuning for the process G p i(s), which is an infinite-order 
system that can be represented in simulation by the following high-order system 
G pl (s)«exp(-Ls)/ng 1 (T i s + l), L=0.01013 (Matausek & Ribic, 2009). Also, it should be 

noted here, that Ziegler and Nichols succeeded seventy years ago in obtaining an excellent 
tuning with the IPDT model defined by K v =0.9251, L=0.1534, which is an extremely crude 
approximation of the real impulse response of the process G p i(s), as in Fig. 3-b. 



Tuning 
method 


k 


h 


kd 


T t 


N d 


IAE 


Mn 


M s 


M p 


optPID 


6.5483 


18.4321 


0.6345 


0.0094 


- 


0.0609 


76.37 


2.00 


1.45 


ZN PID1 


6.9551 


21.8502 


0.5535 


0.0080 


9.980 


0.0538 


76.37 


2.20 


1.72 


ZN PID2 


8.4560 


27.5621 


0.6486 


0.0096 


8.031 


0.0429 


76.37 


2.82 


2.23 


ZN ModifPID2 


6.6450 


21.6592 


0.5097 


0.0073 


10.49 


0.0587 


76.37 


2.16 


1.78 



Table 1. Process G p i(s): comparison of the optimization (optPID) and the Ziegler-Nichols 
tuning in the frequency domain (ZN PID1) and time domain (ZN PID2, ZN ModifPID2). 

The Nyquist curves of G p i(s), G m i(s), and G m 2(s) are almost the same around a u - This is 
important since the PID controller optimization, based on the experimentally determined 
frequency response of the process, under constraints on M s or on M s and M p , is performed 
around the ultimate frequency co a . Amplitudes A and Ao are closely related for the stable 
and unstable processes, as demonstrated in (Matausek & Sekara, 2011) and Appendix. For 
integrating processes A=o u . This means, that the Ziegler-Nichols parameters k u and a u , and 
the Sekara-Matausek parameters cp and A=Ao, for the stable and unstable processes, and 
A=a u , for integrating processes, constitute the minimal set of parameters, measurable in the 
frequency domain, necessary for obtaining PID controller tuning for the desired 
performance/ robustness tradeoff. This will be demonstrated in the subsequent sections. 



3. Optimization of PI/PID controllers under constraints on the sensitivity to 
measurement noise, robustness, and closed-loop system damping ratio 

PID controllers are still mostly used control systems in the majority of industrial 
applications (Desborough & Miller, 2002) and "it is reasonable to predict that PID control 
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Fig. 3. Process G p i(s), denoted as (G p ), and models G m j(s), j=l,2, fc u =11.5919, a u =9.8696, 
r=0.0796 for A=9.0858 (G ml ) and A=A =8.9190 (G m2 ), and G ZN (s)= K v exp(-Ls)/s, K v =0.9251, 
L=0.1534 (ZN): a) step responses, b) impulse responses, c) Nyquist curves of G p i(s) and 
Gzn(s), d) Nyquist curves of G p i(s), G m i(s) and G m 2(s) are almost the same around a u - 
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Fig. 4. Comparison of the optimization and the Ziegler-Nichols (ZN) tuning. Process G p i(s) 
in the loop with the optPID or ZN PID, tuned by using the rules: frequency domain (ZN 
PID1), time domain (ZN PID2), and time domain with the modified proportional gain 
it=0.943/(K v L) (ZN ModifPID2). In all controllers b=0 and D(s)=-5exp(-2.5s)/s. 

will continue to be used in the future" (Astrom & Hagglund, 2001). They operate mostly as 
regulators (Astrom & Hagglund, 2001) and rejection of the load step disturbance is of 
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primary importance to evaluate PID controller performance under constraints on the 
robustness (Shinskey, 1990), measured by the Integrated Absolute Error (IAE). Inadequate 
tuning and sensitivity to measurement noise are the reasons why derivative action is often 
excluded in the industrial process control. This is the main reason why PI controllers 
predominate (Yamamoto & Hashimoto, 1991). However, for lag-dominated processes, 
processes with oscillatory dynamics and integrating/ unstable processes PID controller 
guarantees considerably better performance than PI controller, if adequate tuning of the PID 
controller is performed (Matausek & Sekara, 2011). Moreover, PID controller is a 
"prerequisite for successful advanced controller implementation" (Seki & Shigemasa, 2010). 

Besides PI/ PID controllers, in single or multiple loops (Jevtovic & Matausek, 2010), only 
Dead-Time Compensators (DTC) are used in the process industry with an acceptable 
percentage (Yamamoto & Hashimoto, 1991). They are based on the Smith predictor (Smith, 
1957; Matausek & Kvascev, 2003) or its modifications. However, the area of application of 
PID controllers overlaps deeply with the application of DTC's, as confirmed by the Modified 
Smith Predictor, which is a PID controller in series with a second-order filter, applicable to a 
large class of stable, integrating and unstable processes (Matausek & Ribic, 2012). 

Optimization of the performance may by carried out under constraints on the maximum 
sensitivity to measurement noise Mn, the maximum sensitivity M s and maximum 
complementary sensitivity M p , as done in (Matausek & Ribic, 2012). In this case it is 
recommended to use some algorithm for global optimization, such as Particle Swarm 
Optimization algorithm (Rapaic, 2008), requiring good estimates of the range of unknown 
parameters. Other alternatives, presented here, are recently developed in (Sekara & 
Matausek, 2009, 2010a; Matausek & Sekara, 2011). For the PID controller (3), for F c (s)=l 
defined by four parameters k, ki, kd and T{, optimization under constraints on M n and M s is 
reduced in (Sekara & Matausek, 2009) to the solution of a system of three algebraic 
equations with adequate initial values of the unknown parameters. The adopted values of 
M n and M s are satisfied exactly for different values of £ 2 . Thus, by repeating calculations for 
a few values of the damping ratio of the controller zeros £ z in the range 0.5< £z, the value of 
^corresponding to the minimum of IAE is obtained. Optimization methods from (Sekara & 
Matausek, 2009) are denoted as max(k) and max(ki) methods. 

The improvement of the max(k) method is proposed in (Sekara & Matausek, 2010a). It 
consists of avoiding repetition of calculations for different values of £ z in order to obtain the 
minimal value of the IAE for a desired value of M s . In this method, denoted here as method 
optPID, the constrained optimization is based on the frequency response of model (5). 

For the PI optimization, an improvement of the performance/ robustness tradeoff is 
obtained by applying the combined performance criterion / c =/#Ci+(l -p)a> (Sekara & 
Matausek, 2008). Thus, one obtains 

max ] c , (13) 

k if ai 

F(»,fc,fc,-) = , dF(a r k,ki) /dco = r (14) 

where 0<oo<co and /? is a free parameter in the range 0</3<l. The calculations are repeated for 
a few values of /?, in order to find /? corresponding to the minimum of IAE. The optimization 
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in this method, denoted here as opt2, is performed for the desired value of M s . For /?=1 one 
obtains the same values of parameters k and k\ as obtained by the method proposed in 
(Astrom et al., 1998), denoted here as optl. 

The most general is the new tuning and optimization procedure proposed in (Matausek & 
Sekara, 2011). Besides the tuning formulae, the optimization procedure is derived. For the 
PID and PI controllers it requires only obtaining the solution of two nonlinear algebraic 
equations with adequate initial values of the unknown parameters. PID optimization is 
performed for the desired closed-loop system of damping ratio £ and under constraints on 
M n and M s . Thus, for £=1 the critically damped closed-loop system response is obtained. PI 
optimization is performed under constraint on M s for the desired value of £. The procedure 
proposed in (Matausek & Sekara, 2011) will be discussed here in more details, since it is 
entirely based on the concept of using oscillators (6)-(7) for dynamics characterization of the 
stable processes, processes having oscillatory dynamics, integrating and unstable processes. 
The method is derived by defining a complex controller C(s)=lc u (l+C(s)), where the 
controller C"(s), given by 

C ' (S) = A + M\1 F ,f (S) ( A wW ' E(S) = ^ + W + X A(S) = * V + 2a$ + 1 ' (15) 

Am u A(s) 1 - E(s)exp(-rs) / A(s) 

is obtained by supposing that in Fig. 1 process G p (s) is defined by oscillator G*(s) in (6), 

approximated by (7). Complex controller C(s)=/c u (l+C*(s)) is defined by the parameters k w 
On, r, A and by the two tuning parameters X and £, with the clear physical interpretation. 
Parameter X is proportional to the desired closed-loop system time constant. Parameter £ is 
the desired closed-loop system damping ratio. Then, by applying Maclaurin series 
expansion, the possible internal instability of the complex controller C(s) is avoided and 
parameters of PID controller C(s) in Fig. 1 are obtained, defined by: 

j _ V2-p2{Vl-Pl)-P?,+ 1 / a >l (1(>) 

( p t -r h -{l-M n /\k a \)/fi l ' 

k = k u (/3 1 (T !+t]l -j3 2 ) + l),k i =k u /3 1 , (17) 

h = KAfa+Vt-fiiXm -M-fr+l/aQ+kJf ( 18 ) 

Parameters rji, rjz, fii, fii and j?3, from (Matausek & Sekara, 2011), depends on X, £ and k u , a u , t, 
A. They are given in Appendix. Generalization of this approach is presented in (Sekara & 
Trifunovic, 2010; Sekara et al., 2011). 

For the desired closed-loop damping ratio £=1, A=l/o)u, and for 

T f =l/(NcoJ, (19) 

one obtains (Matausek & Sekara, 2011) the PID tuning that guarantees set-point and load 
disturbance step responses with negligible overshoot for a large class of stable processes, 
processes with oscillatory dynamics, integrating and unstable processes. Tuning formulae 
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defined by (17)-(19) are denoted here as method tun\ u . Absolute value of the Integrated 
Error (IE), approximating almost exactly the obtained IAE, is given by | IE | =l/( |/c u |/?i). 
Here the value Tf=l/(10<a u ) is used, as in (Matausek & Sekara, 2011). 

To demonstrate the relationship between PID controller, tuned by using the method tunAu, 
and complex controller C(s)=k u (l+C'(s)), obtained for X=l/a u and £=1, the frequency 
responses of these controllers, tuned for the process 



G p2 (s) = l/(s + T) 4 



(20) 



are presented in Fig. 5-a. For this process, parameters k u , a>u, r, A, p and cp are given in 
Appendix. The load disturbance unite step responses, obtained for G P 2(s) in the loop with 
the PID controller and complex controller C(s), are presented in Fig. 5-b. Further details 
about the relationship between these controllers are presented in (Matausek & Sekara, 2011; 
Trifunovic & Sekara, 2011; Sekara et al., 2011). 




/ V k u (HC(s)) 


/ ^*-»w Lx piD 


! 
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Fig. 5. Comparison of the complex controller C(s)=fc u (l+C*(s)) with PID controller, both 
tuned for G p 2(s): a) Bode plots of the controllers and b) the load unite step disturbance 
responses of G p 2(s) in the loop with these controllers. 

By applying tuning formulae (17)-(19), the desired closed-loop damping ratio £=1 is 
obtained with the acceptable values of maximum sensitivity M s and maximum sensitivity to 
measurement noise M n . However, when a smaller value of M n is required for a desired 
value of M s and the desired closed-loop damping ratio £ the other possibility is to determine 
the closed-loop time constant X and the corresponding ao, by using (16)-(18) and by solving 
two algebraic equations: 



|l + C(iw)G m (i«)| -1/M S Z =0, 



(21) 



S(|l +C(ia)G m (ia)\ 2 ) J da> = . 



(22) 



In this case, the PID controller in (3), Fc(s)=l, is obtained for the desired critical damping 
ratio £=1 of the closed-loop system and the desired values of M n and M s . This is the unique 
possibility of the procedure (16)-(18) and (21)-(22) proposed in (Matausek & Sekara, 2011). 
Moreover, by repeating the calculations for a few values of £ the value of £ is obtained 
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guaranteeing, for desired M n and M s , almost the same value of the IAE as obtained by the 
constrained PID optimization based on the exact frequency response G p (ia>). This PID 
optimization method is denoted here as the method opt2A, when the quadruplet {k u , a>uj <p, 
A) is used, or opt2Ao, when the quadruplet {k u , a n , cp, Aoj is used. It should be noted here, 
that for fcd=0 and Tf=0, by relations (17) and (21)-(22) a new effective constrained PI 
controller optimization is obtained, denoted here as opt3. It is successfully compared 
(Matausek & Sekara, 2011) with the procedure proposed in (Astrom et al., 1998), optl. 

Now, tuning defined by (17)-(19) with N=10, X=l/a u and £=1, method tunXu, will be 
compared with the optimization defined by (16)-(18), (21)-(22), method opt2A. Both 
procedures guarantee desired critical damping £=1, however only the second one 
guarantees the desired values of M n and M s . Thus, for £=1 and for the maximum sensitivity 
Ms obtained by applying method tunXu, the smaller value of sensitivity to measurement 
noise M n will be used by applying PID optimization method opt2A. The results of this 
analysis are presented in Table 2 and Fig. 6. As in Table 1, controller is tuned by using the 
model G m (s) in (5) and then applied to processes G p 3(s) to obtain IAE, M s and M p , where 



G p3 (s) = 



1.507(3.42s + !)(!- 0.816s) 



(577s + l)(18.1s + l)(0.273s + l)(104.6s / + 15s + 1) 



(23) 



Lower value of IAE is obtained, for almost the same robustness, by using higher value of the 
sensitivity to measurement noise. However, for the lower value of M n the controller and, as 
a result, the actuator activity is considerably reduced. Thus, the comparison of the IAE, 
obtained by the PID controllers with the same robustness, is meaningless if the sensitivity to 
measurement noise M n is not specified, as demonstrated in Fig. 6. This fact is frequently 
ignored. 



method 


A 


k 


h 


k d 


Ti 


IAE 


Mn 


Ms 


M p 


tun\ u 


17.3310 


22.3809 


0.2778 


377.2723 


1.7331 


3.62 


217.7 


2.14 


1.58 


opt2A 


20.4849 


18.2791 


0.1944 


345.5996 


5.0893 


5.17 


67.91 


2.12 


1.51 



Table 2. Process G p 3(s) in the loop with the PID controllers. Tuning method (17)-(19), tun\ u 
and optimization (16)-(18), (21)-(22), opt2A for £=1. 

Concluding this section, the constrained PI/PID controller optimization methods proposed 
in (Matausek & Sekara, 2011) is compared with the constrained PID controller optimization 
method proposed in (Sekara & Matausek, 2010a), optPIDl, and the constrained PI controller 
optimization method proposed in (Sekara & Matausek, 2008), opt2. The test batch of stable 
processes, processes having oscillatory dynamics, integrating and unstable processes used in 
this analysis is defined by transfer functions G p i(s), G p 2(s), G p 3(s) and 



G p4 (s) = 



(s + 1) 3 



C p5 (s)- 



9s 1 + 0.24s + 1 



(24) 



G p6 (s) = 



s(s + l)(0.5s + l)(0.25s + l)(0.125s + 1) 



,G p7 (s)- 



2e' 



(10s-l)(2s + l) 



(25) 
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with parameters k m a>a, r, A, Aq, p, <p presented in Appendix. Comparison of the methods for 
PID controller tuning is presented in Table 3. Comparison of the methods for PI controller 
tuning is presented in Table 4 and Fig. 7. 
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Fig. 6. Set-point, R(s)=l/s, and load disturbance, D(s)=-10exp(-400s)/s, step responses. G P 3(s) 
and PID controllers tuned by: a) tunAu, b=0.5; b) opt2A, b=0.6. Measurement noise is 
obtained by passing uniform random noise ±1 through a low-pass filter F(s)=0.5/(10s+l). 



Process/ 
method 


k 


k 


fed 


Tt 


ME 


M n 


Ms 


M p 


& 


^ _ 


G p3 /max(k) 


17.0778 


0.2372 


320.06 


4.7131 


4.83 


67.91 


2.00 


1.56 


0.98 


- 


G p3 /optPID 


17.1037 


0.2303 


315.14 


4.6407 


4.84 


67.91 


2.00 


1.54 


- 


- 


G p3 /opt2A 


17.1994 


0.1788 


316.59 


4.6621 


5.62 


67.91 


2.00 


1.41 


- 


1 


G p3 /opt2Ao 


16.9411 


0.2670 


312.65 


4.6040 


4.87 


67.91 


2.00 


1.69 


- 


0.75 


G p3 /opt2A 


16.8802 


0.2083 


268.32 


3.9513 


4.92 


67.91 


2.00 


1.59 


- 


0.80 


G p5 /max(ki) 


-0.3090 


0.0654 


0.8640 


1.7597 


21.17 


0.49 


2.00 


1.03 


0.65 


- 


G p5 /optPID 


-0.3032 


0.0651 


0.8280 


1.6864 


21.87 


0.49 


2.00 


1.07 


- 


- 


G p5 /opt2A 


-0.4139 


0.0336 


0.9398 


1.9140 


30.04 


0.49 


2.00 


1.04 


- 


1 


G p s/opt2Ao 


-0.3369 


0.0583 


0.8948 


1.8223 


20.29 


0.49 


2.00 


1.02 


- 


0.65 


G p5 /opt2A 


-0.3542 


0.0528 


0.8860 


1.8044 


20.30 


0.49 


2.00 


1.02 


- 


0.70 


G p6 /max(k) 


0.1177 


0.0063 


0.3961 


0.8353 


207.22 


0.47 


2.00 


1.76 


1.18 


- 


G p6 /optPID 


0.1181 


0.0054 


0.3736 


0.7878 


208.65 


0.47 


2.00 


1.63 


- 


- 


G p6 /opt2A 


0.1133 


0.0043 


0.2373 


0.5003 


234.73 


0.47 


2.01 


1.60 


- 


1 


G p6 /opt2A 


0.1160 


0.0043 


0.2709 


0.5712 


233.50 


0.47 


2.01 


1.55 


- 


1.05 


G P 7/max(k) 


0.8608 


0.0158 


3.3101 


0.1418 


73.50 


23.35 


3.61 


3.39 


1.88 


- 


G p7 /optPID 


0.8609 


0.0150 


3.2946 


0.1411 


75.00 


23.35 


3.61 


3.33 


- 


- 


G p7 /opt2A 


0.8543 


0.0106 


2.9385 


0.1258 


93.96 


23.35 


3.54 


3.18 


- 


1.3 


G p7 /opt2A 


0.8060 


0.0093 


2.3759 


0.1017 


107.41 


23.35 


3.61 


3.77 


- 


1.1 



Table 3. PID controllers, obtained by applying model G m (s) and tuning methods: max(k), 
max(ki); (31)-(35) optPID; (16)-(18), (21)-(22) opt2A and opt2A . 

In Table 3 optimization (16)-(18), (21)-(22) is performed for stable G p 3(s), Gps(s) and unstable 
G P 7(s) processes by using G m (s) with two quadruplets: {k u , (Oa, <p, A], denoted as opt2A, and 
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{kw (On, <P, Ao], denoted opt2Ao. As mentioned previously, for integrating processes A=co u . 
Almost the same performance/ robustness tradeoff is obtained for A and Ao, as supposed in 
Section 2. This result is important since it confirms that an adequate approximation of the 
frequency response of the stable and unstable processes around Wa can be used in the 
optimization (16)-(18) and (21)-(22), instead of the model G m (i&)) in (5). Obviously, the same 
applies for integrating processes. The advantage of the constrained PID controller 
optimization (16)-(18) and (21)-(22) is that only two nonlinear algebraic equations have to be 
solved, with very good initial conditions for the unknown parameters X and coo. Moreover, 
the optimization is performed for the desired values of M s , M n and for the desired closed- 
loop system damping ratio f. 

Finally, the results of the PI controller optimization are demonstrated in Table 4 and in Fig. 
7. By repeating calculations for a few values of £, for the same values of M s and M p , the same 
(minimal) value of the IAE is obtained by applying method opt3, defined by (17) and (21)- 
(22), and the method opt2, defined by (13)-(14). As mentioned previously, method opt2 is an 
improvement of the method proposed in (Astrom et al., 1998), denoted here as method optl. 
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Process/method 


k 


h 


IAE 


Ms 


M p 


P 


c 


Gpi/optl 


2.6707 


6.4739 


0.19 


1.98 


1.58 


1 


- 


G p i/opt2 


3.1874 


6.1391 


0.16 


1.99 


1.48 


0.52 


- 


G p i/opt3 


3.2119 


6.1083 


0.16 


1.99 


1.48 


- 


0.95 


G p3 /optl 


7.4060 


0.0692 


15.61 


1.92 


1.65 


1 


- 


G p3 /opt2 


8.1456 


0.0680 


14.72 


1.94 


1.60 


0.48 


- 


G p3 /opt3 


8.1355 


0.0679 


14.73 


1.94 


1.60 


- 


0.90 


G p4 /optl 


0.3248 


0.1259 


12.04 


2.16 


1.35 


1 


- 


G p4 /opt2 


0.4608 


0.1137 


10.23 


2.11 


1.18 


0.69 


- 


G p4 /opt3 


0.4651 


0.1128 


10.19 


2.10 


1.18 


- 


0.90 



Table 4. PI controllers, obtained for M s =2 by applying model (5) and methods: (Astrom et 
al., 1998) optl, (13)-(14) opt2, and (17), (21)-(22) opt3. 

4. Closed-loop estimation of model parameters 

Approximation of process dynamics, around the operating regime, can be defined by some 
transfer function G p (s) obtained from the open-loop or closed-loop process identification. 
One two step approach (Hjalmarsson, 2005) is based on the application of the high-order 
ARX model identification in the first step. In the second step, to reduce the variance of the 
obtained estimate of frequency response of the process, caused by the measurement noise, 
this ARX model is reduced to a low-order model G p (s). By applying this procedure an 
adequate approximation G p (ie>) of the unknown Nyquist curve can be obtained in the region 
around the ultimate frequency o n . As demonstrated for the Ziegler-Nichols tuning, in Fig. 3- 
c and Fig. 4-b, such approximation of the unknown Nyquist curve is of essential importance 
for designing an adequate PID controller. The same applies for the successful PID 
optimization under constraints on the desired values of M n and M s , as demonstrated in 
Table 5 for the value of A defined as in (5) and for A=Aq. 

The Closed-Loop (CL) system identification can be performed by using indirect or direct 
identification methods. In indirect CL system identification methods it is assumed that the 
controller in operation is linear and a priory known. Direct CL system identification 
methods are based only on the plant input and output data (Agiiero et al., 2011). Finally, the 
identification can be based on the simple tests, as initiated by Ziegler and Nichols (1942), to 
obtain an IPDT model (1). Later on, this approach is extended to obtain FOPDT model and 
the Second-Order Plus Dead-Time (SOPDT) model, for integrating processes characterized 
by the IFOPDT model. The SOPDT model can be obtained from k u , a u , cp, A. In this case it is 
defined by 



G SO (s) ; 



as 1 + bs + c ' 



(26) 



where parameters a, b, c and L are functions of k a , ffl w cp and A, obtained from the tangent 
rule (Sekara & Matausek, 2010a). This model (26) is an adequate SOPDT approximation of 
the Nyquist curve G p (i<») in the region around the ultimate frequency (0 W for a large class of 
stable processes, processes with oscillatory dynamics, integrating and unstable processes. 
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The recently proposed new Phase-Locked Loop (PLL) estimator (Matausek & Sekara, 2011), 
its improvement (Sekara & Matausek, 2011c), and new relay SheMa estimator (Sekara & 
Matausek 2011b) make possible determination of parameters k u , a^ (p and Ao of the model 
G m (s) in the closed-loop experiments, without breaking the control loop in operation. This 
property of the proposed PLL and SheMa estimators is important for practice, since 
breaking of control loops in operation is mainly ignored by plant operators, especially in the 
case of controlling processes with oscillatory dynamics, integrating or unstable processes. 
The PLL estimator can be applied in the case when the controller in operation is an 
unknown linear controller, while the SheMa estimator can be applied when the controller in 
operation is unknown and nonlinear. In that sense, the SheMa estimator belongs to the 
direct CL system identification methods, based only on the plant input and output data, as 
in (Agiiero et al., 2011). Both procedures, SheMa and PLL, are based on the parameterization 
presented in (Sekara & Matausek, 2010a; Matausek & Sekara, 2011). Estimates of 
parameters k u , k u , k^ and <» u , a> u , w^ , obtained for argG p (ia>) = -7i + <p, <p = and 

cp = (p~ = +;r/36 , are used for determining <p and Ao, as defined in (Matausek & Sekara, 
2011). 

In this section, an improvement of the new PLL estimator from (Matausek & Sekara, 2011) is 
presented in Fig. 8. The improvement, proposed by Sekara and Matausek (2011c), consists of 
adding two integrators at the input to the PLL estimator from (Matausek & Sekara, 2011). 
Inputs to these integrators are defined by outputs of the band-pass filters AFi, used to 
eliminate the load disturbance. Outputs of these integrators are passed through a cascade of 
the band-pass filters AF m , m=2,3,4. All filters AF m , m=l,2,3,4, are tuned to the ultimate 
frequency. Such implementation of the PLL estimator eliminates the effects of the high 
measurement noise and load disturbance. Blocks AF m , j=l,2,3,4, are implemented as 
presented in (Matausek & Sekara, 2011), while implementation of blocks for determining 
argjGp(ici)} and | G p (i<») | are presented in (Sekara & Matausek, 2011c). 

PLL estimator from Fig. 8 is applied to processes G p s(s)=exp(-s)/(2s+l) and 
G p 9(s)=4exp(-2s)/(4s-l) in the loop with the known PID controller. Estimation of parameters 

k~, k u , k^ and m n , m u , o^ is presented in Fig. 9. Highly accurate estimates of k u , k u , k^ 
and a> u , a> n , a* are obtained in the presence of the high measurement noise and load 
disturbance. Since these parameters are used to determine cp and Ao, this experiment 
demonstrates that highly accurate estimate of the quadruplet {k u , o^, cp, Ao} can be obtained, 
in the presence of the high measurement noise and load disturbance, by the PLL estimator 
from (Sekara & Matausek 2011c). In Fig. 10, estimation of the unknown Nyquist curve of the 
unstable process in the loop with the PID controller is demonstrated. 

The PLL estimator from (Matausek & Sekara, 2011; Sekara & Matausek 2011c) is a further 
development of the idea firstly proposed in (Crowe & Johnson, 2000) and used in (Clarke & 
Park, 2003). The SheMa estimator is a further development of the estimator proposed by 
Astrom and Hagglund (1984) as an improvement of the Ziegler-Nichols experiment. 

The Ziegler and Nichols (1942) experiment, used to determine k u and a>a of a process is 
performed by setting the integral and derivative gains to zero in the PID controller C(s) in 
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operation. However, in this approach the amplitude of oscillations is not under control. This 
drawback is eliminated by Astrom and Hagglund (1984). The factors influencing the critical 
point estimation accuracy in this conventional relay setup are: the use of describing function 
method is faced with the fact that higher harmonics are not efficiently filtered out by the 
process, presence of the load disturbance d, and presence of the measurement noise n. The 
first drawback of the conventional relay experiment is eliminated by the modified relay 
setup (Lee et al., 1995). 
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Fig. 8. Improved PLL estimator. AF2,3,4 is the cascade of band-pass filters AF„„ m=l,2,3,4. 
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Fig. 9. PLL estimates of k u , k u , k^ and eo u , a> u , m^, in the presence of the high 
measurement noise and step load disturbance at f=700 s. Process Gps(s)=exp(-s)/(2s+l), for: 
f = -j[ J 36 for 0<f<300 s, </> = for 300<f<500 s and f = it / 36 for 500<£<1000 s . 
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Nyquist Diagram 




Fig. 10. Estimates (circles) of the Nyquist curve (solid) obtained by the PLL estimator for the 
desired values &ef= ar g{G P 9(i&>)}. Process G p g(s)=4exp(-2s)/(4s-l), the noise-free case. 

Due to its simplicity, the relay-based setup proposed by Astrom and Hagglund (1984) is 
still a basic part of different methods developed in the area of process dynamics 
characterization. For example, it is used to generate signals to be applied for determining 
FOPDT and SOPDT models, using a biased relay (Hang et al., 2002). However, from the 
viewpoint of the process control system in operation, the estimation based on this setup, and 
its modifications, is performed in an open-loop configuration: the loop with the controller 
C(s) in operation is opened and the process output is connected in feedback with a relay. 

In the paper (Sekara & Matausek, 2011b) a new relay-based setup is developed, with the 
controller C(s) in operation. It consists of a cascade of variable band-pass filters AF„„ from 
(Clarke & Park, 2003), a new variable band-pass filter F mo d proposed by Sekara and 
Matausek (2011b) and a notch filter FNF = l-Fmod- Center frequencies of variable band-pass 
filters AF„, and F mo d are at a>a- 

Highly accurate estimates of o)a and k u are obtained in the presence of the measurement 
noise and load disturbance. Also, highly accurate estimates of the Nyquist curve G p (i(y) at 
the desired values of arg{G p (iffi)} are obtained by including into the SheMa the modified 
relay instead of the ordinary relay. The amplitude j,i of both relays is equal to ^=7rfc u ,oi/ref£o/4, 
where k u ,o is the ultimate gain obtained in the previous activation of the SheMa, y m { is the 
amplitude of the set-point r and £o is a small percent of i/ re f, for example £o=0.1% in the 
examples presented in (Sekara & Matausek, 2011b). The proposed closed-loop procedure 
can be activated or deactivated with small impact on the controlled process output. Further 
details of the SheMa estimator, including the stability and robustness analyses, and 
implementation details, are presented in (Sekara & Matausek 2011b). 



5. Gain scheduling control of stable, integrating, and unstable processes, 
based on the controller optimization in the classification parameter plane 

For a chosen region in the p-cp classification plane, presented in Fig. 11, the normalized 
parameters k n (p,(p), h n (p,(p), kd n (p,<p) and Tf n = \kd n {p,(p) \/m n of a virtual PID n controller are 
calculated in advance by using the process-independent model G n (i(» IV p, (p) in (10). 
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Then, parameters k, ki, kd and T{ of the PID controller (3), Fc(s)=l, are obtained, for the 
process classified in the chosen region of the p-cp plane, by using the estimated k u , a w cp, A 
and the following relations 



: ^u^n' 



k 



■ K a u k in> 



z K k dn I <»u> 



T l = T ln I ®u ■ 



(27) 



Depending on the method applied to obtain parameters k m km, kdn and T m = \ kdn \ /m n of a 
PID n controller, parameters k, k\, kd and Tt of the PID controller (3), Fc(s)=l, guarantee the 
desired M s and the sensitivity to measurement noise equal to M n = | k u \ m n , or guarantee the 
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Fig. 11. Classification p-cp parameter plane, with processes Gpj(s),/=1,2,...,9. Stable processes 
are classified in the region Q < p <\,Q < <p < n / ^p + 1 , integrating processes are classified as 
p = l, Q <(p<K / \j2 processes. Unstable processes are classified outside this region. 

desired Ms, £, and M n = | k u \ m n - Since parameters k lv km, kdn and Tt n = \ kdn \ /m n are determined 
in advance, they can be memorized as look-up tables in the p-cp plane. Besides, this can be 
done for different values of M s , m n and £. These look-up tables define a new Gain 
Scheduling Control (GSC) concept. Important feature of this GSC is that these look-up 
tables, obtained for some values of M s , m n and £ from the model G n (i<an, p, <p), are process- 
independent. Enormous resources are avoided, required for performing experiments on the 
plant in order to define the standard GSC as the look-up tables of PID controller parameters 
for this plant and the desired region of operating regimes. Thus, the important and exclusive 
feature of the new GSC is that a desired performance/ robustness tradeoff can be obtained 
for a large region of dynamic characteristics of processes in different plants and different 
operating regimes, covered by the look-up tables of parameters k n , k m , kdn in the p-cp 
classification plane. 

Now, this GSC PID controller tuning, performed by using (27), will be demonstrated by the 
two different procedures applied for obtaining parameters k lv kin, kdn and T m = \kdn\/m n of 
the PID n controller for integrating and stable processes. Stable processes having a weakly 
damped impulse response are denoted as processes having oscillatory dynamics, while 
processes with damped impulse response are denoted as stable processes. 

For integrating processes, parameters k n , k m , kdn and T m = \ kdn\ /m n of the PID n controller 
depend only on angle cp, since p=l. In this case, for desired values of M s and mn, PID 
controller parameters (27) are obtained from tuning formulae for k n ((p), ki n (cp) and kdn{f>) 
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(Sekara & Matausek, 2011a). Thus, for integrating process G p g(s) parameters of the PID n 
controller are obtained by applying angle cp=0.9716 in the tuning formulae defined in 
(Sekara & Matausek, 2011a) for M s =2 and m n =2, given in Appendix as tunl. The results are 
presented in Table 5, G p 6-tunl. 

For processes having the oscillatory dynamics look-up tables and tuning formulae are derived 
in (Sekara & Matausek, 2011a) for M s =2 and m n =40, in the region 0.1<p<0.2, 0.1745<<p<1.0472 of 
the p-cp classification plane of Fig. 11. These tuning formulae, in Appendix denoted as tun2, are 
applied to determine parameters k, kj, k& and Tt for the process having the oscillatory dynamics 
Gps(s), classified as process p=0.1971, cp=0.3679 (Table 5, G p s-tun2). To illustrate the direct 
application of the look-up tables from (Sekara & Matausek, 2011a, Table A4) and interpolation 
procedure defined in Appendix, Fig. 17, since this process is classified as p=0.1971, <p=21.0791 
(0.3679), the following points are determined from (Sekara & Matausek, 2011a, Table A4) and 
Appendix, Fig. 17: pi,i=0.15, ipi,i=20 , pi,2=0.2, ipi,2=20 and p2,2=0.2, if>2,2=30 . Parameters (k n km, 
fedn) are defined by: (-2.4122, 0.5988, 3.9353) for p u , cp u , (-1.7022, 0.4125, 2.8783) for fh,bfi,2 and 
(-1.6626, 0.4164, 2.3017) for p2,2,cpi,2- Then, by using three point interpolation from Appendix, 
upper triangle (a ra =0.0578, /? ra =0.1971), one obtains parameters in Table 5, G p 5-GSC: fc=-0.4220, 
fci=0.0384, fc d =1.9116, T f =1.947. 

For stable processes, in a large region of the p-cp plane, look-up tables of parameters k n h n 
and kdn are defined for M s =2 and m n =2 (Sekara & Matausek, 2011a, Tables A1-A3). These 
look-up tables are applied in the present paper to determine parameters k, k\, kd and T{ for 
the stable process G p 3(s). This process is classified as process p=0.9808, ip=0.6783 (38.8637 ). 
Thus, for G p 3(s) parameters {k n , k m , fcdn) can be obtained from the three points in the p-cp 
classification plane (Appendix, Fig. 17): j0i,i=0.95, (pi,i=30 ; /J2,i=0.95, if>2,i = 40 and p2,2=l, 
ip2,2 = 40 (0.6981). Two points are used for stable processes (0.5086, 0.1349, 0.6569) for pi,i,<pi,i 
and (0.5013, 0.1261, 0.5332) for p 2 ,i,^2,i from the look-up tables (Sekara & Matausek, 2011a, 
Tables A1-A3), while data (0.5036, 0.1109, 0.5332) for p2,2. 92,2 are obtained from tuning 
formulae derived for integrating processes in (Sekara & Matausek, 2011a), given in 
Appendix as tunl. Then, by using three point interpolation from Appendix, Fig. 17 lower 
triangle (fl«=0.6166, /?//=0.1136), one obtains parameters presented in Table 5, G p 3-GSC: 
fc=17.0973, fci=0.2307, fc d =315.2928 and T f =4.6430. 



Process-method 
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k d 


T{ 


IAE 


Mn 


Ms 


M p 


G P 3-GSC 


17.0973 


0.2307 


315.29 


4.6430 


4.84 


67.91 


2.00 


1.54 


G p s-tun2 


-0.4220 


0.0380 


1.8758 


0.1910 


26.32 


9.82 


1.99 


1.08 


G P 5-GSC 


-0.4269 


0.0384 


1.9116 


0.1947 


26.04 


9.82 


2.01 


1.09 


G p6 -tunl 


0.1182 


0.0054 


0.3746 


0.7970 


209.10 


0.47 


2.00 


1.62 



Table 5. PID controllers: stable process G P 3(s), method GSC-Appendix; stable process having 
oscillatory dynamics G p s(s), method tun2 and method GSC-Appendix; integrating process 
G P 6(s), method tunl. 



5.1 Experimental results 

Experimental results, presented in Fig. 12, are obtained by using the laboratory thermal 
plant. It consists of a thin plate made of aluminum, L a =0.1m long and J?=0.03m wide 
(Matausek & Ribic, 2012). Temperature T(x,f) is distributed along the plate, from x=0 to 
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x=L a/ and measured by precision sensors LM35 (T092), at x=0 and x=L a . The plate is heated 
by a terminal adjustable regulator LM317 (TO 220) at position x=Q. The manipulated 
variable is the dissipated power of the heater at x=0. The input to the heater is the control 
variable u(t) (%), defined by the output of the PID controller. The controlled variable is 
y(t)=T(L a ,t), measured by the sensor at position x=L a . Temperature sensor at x=0 is used in 
the safety device, to prevent overheating when 70 C <T(0,f). The anti-windup 
implementation of the PID controller (3), Fc(s)=l, is given by 



( 



bks + fcj 



k d s +ks + k i 
(T aw s + l)(T fS + l) 



T„.,s + 1 



(28) 



The saturation element is defined by the input wc(f) and output w(f): 



M c ^ 'low 
'low <U C < /jugh 



(29) 



['high' 



«r ^ hi 



/ligh 



Obviously, in the linear region /iow < wc(f) < 'high of the saturation element, for uc(t)=u(t) one 
obtains (3), F c (s)=l, from (28). 
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Fig. 12. Experimental results. Set-point and load step (-20% change of the controller output 
at t=1600 s) responses of the real plant, with the PI and PID controller: a) control variable 
u(t) and b) controlled variable y(t). The real plant, with the anti-windup PID controller 
under the disturbance induced by activating/ deactivating the fan: c) u(t) and d) y(t). 
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Transfer function G p 3(s), used for determining parameters of the PID controller applied in 
the real-time experiment, is obtained previously in (Matausek & Ribic, 2012). By applying a 
Pseudo-Random-Binary-Sequence for u(t), the open-loop response y(f) of the laboratory 
thermal plant is obtained. From these u(t) and y(f) a 100-th order ARX model is determined 
and reduced then to the 5-th order transfer function G p 3(s) in (Matausek & Ribic, 2012). This 
model of the process is used here to determine the quadruplet \k u , a>a, cp, A] presented in the 
Appendix. Thus, the laboratory thermal plant is classified as the process p=0.9808, cp=0.6783. 
Then, PID controller applied to the real thermal plant is determined by using look-up tables 
of parameters k n (p,cp), ki n (p,(p), kd n (p,cp), for stable processes, and parameters k n (<p), k\ n {cp), 
kdn((p), for integrating processes, previously determined in (Sekara & Matausek, 2011a). This 
procedure, used to obtain PID in Table 5, row G p 3-GSC, and results obtained by this PID 
controller, presented in Fig. 12, demonstrate that in advance determined look-up tables of 
parameters k n , k m and fcdn defines a process-independent GSC applicable for obtaining the 
desired performance/ robustness tradeoff for a real plant classified in the p-cp parameter 
plane. For Ti=k/ki and Td=kd/k, parameter T aw =15s is obtained from T !iW =pT i +(l-p)Td, for 
p=0.2, and Zi ow =0, /h ig h=100%, b=0.25. 

Closed-loop experiment in Fig. 12-a and Fig. 12-b is used to demonstrate advantages of the 
designed PID controller, compared with the PI controller, from Table 4, row G P 3/opt3 
defined by: fc=8.1355, fci=0.0679, and b=0.5. This experiment starts from temperature 
T(L a ,f)a45 C, as presented in Fig. 12-b. Then at f=1000 s the set point is changed to r=45 C+ro, 
ro=5 C. At £=1600 s a load disturbance is inserted as a step change of the controller output 
equal to -20%. Improvement of the performance obtained by the PID controller is evident. 
As expected, this is obtained with the greater variation of the control signal Mpio(f) than that 
obtained by Mpi(f). This is the reason why PID controller from Table 2, row tunAu, having a 
greater value of M n =217.7, is not applied to the real thermal plant. 

The closed-loop experiment presented in Fig. 12-c and Fig. 12-d starts from the steady state 
temperature T(La,f)«50 C by activating a fan at £=400 s. Then, at £=600 s the fan is switched-off. 
Action of the fan induced a strong disturbance, as seen from the control signal u(t) in Fig. 12-c. 
It should be observed that anti-windup action is activated two times, around 410 s and 625 s. 
Anti-windup action is effective and rejection of the disturbance is fast, as seen from Fig. 18-d. 

6. Conclusion 

The extension of the Ziegler-Nichols process dynamics characterization, developed in 
(Sekara & Matausek, 2010a; Matausek & Sekara, 2011), is defined by the model (5). Based on 
this model, a procedure is derived for classifying a large class of stable, integrating and 
unstable processes into a two-parameter p-cp classification plane (Sekara & Matausek, 2011a). 
As a result of this classification, a new CSC concept is developed. In the p-cp classification 
plane, parameters g n (p,cp)={k n (p,cp), h n (p,cp), k dn (p,cp)} and Tin(p,cp)= \kdn(p,cp) \/m IU of a virtual 
PID n controller can be calculated in advance, to satisfy robustness defined by M s and 
sensitivity to measurement noise defined by m n . Also it is possible to satisfy M s , m n and the 
closed-loop system damping ratio £. Calculation of parameters gn(p,cp) and Ti n (p,cp) is 
process-independent. The calculation is performed by using model G n (s n/ p,^i), defined by the 
values of p and cp for stable processes in the range < p < 1, Q <cp<n / yjp + 1 , for 
integrating processes in the range p = l, <cp<7t / \j2 , for unstable processes by the values 
of the p and cp outside these regions. 
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Parameters gn(p,<p), calculated for a given region in the p-cp classification plane, are 
memorized as process-independent look-up tables. Then, for the process G p (s) classified into 
this region of the p-cp classification plane, parameters of a real PID controller k, k^ k&, T{ are 
obtained directly from g n (p/<p), T( n (p,(p) and the estimated quadruplet {k u/ co u , (p, A) or {k u/ co u , 
q>, Ao) for stable/ unstable processes, and the triplet {k^ a>u, (p) for integrating processes. It is 
demonstrated by simulations that for the real M n equal to M n = | k u \ m m the desired M s and £ 
are obtained when a real PID controller, obtained by the proposed GSC, is applied to the 
process G p (s). The desired performance/ robustness tradeoff can be accurately predicted. 
Namely, performance index IAE and robustness index M s , obtained on the model G m (s) in 
(5) are almost the same as those obtained for the process G p (s), as confirmed here and by a 
large test batch considered in (Sekara & Matausek, 2010a; Matausek & Sekara, 2011; Sekara 
& Matausek, 2011a). 

A set of new constrained PID optimization techniques is derived for determining the four 
parameters k, ky k&Tt of the PID controller. The one of them has a unique property. The 
unknown parameters are obtained as the solution of only two nonlinear algebraic equations, 
with the good initial values of the unknown two parameters, determined to satisfy the 
desired values M s and Mn, given desired value of the closed-loop system damping ratio £. 
Thus, the critically damped closed-loop system response is obtained for £=1, Two extensions 
of the PLL-based and relay-based procedures are derived in (Matausek & Sekara, 2011; 
Sekara & Matausek, 2010b; 2011c; 2011b) for determining the quadruplet {k w cou, <p, Ao}. 
These procedures can be applied for the closed-loop PID controller tuning/ retuning, in the 
presence of the measurement noise and load disturbance, without breaking the loop of the 
controller in operation. 

Process-independent look-up tables of parameters gn(p,<p), defining the process-independent 
GSC, can be applied by using any process dynamics characterization defined by the 
estimated frequency response of the process around the ultimate frequency. This is 
demonstrated in the present chapter by applying a model obtained previously by a high- 
order ARX identification of a laboratory thermal plant, and reduced then to the fifth order 
G p 3(s), used here to determine the quadruplet {k m co u , <p, A). This quadruplet is applied to 
determine parameters of the real PID, by using the look-up tables of parameters gn(p,(p) 
calculated previously in (Sekara & Matausek, 2011a). As confirmed by the experimental 
results, the method of the proposed process-independent GSC is effective. Finally, it is 
believed that material presented in this chapter will initiate further development of the 
proposed process-independent GSC and its implementation in advanced controllers. 

7. Appendix 

Parameters r\\, r]2, §i, §2 and §3 

_ a x sin(&> u r) + a 2 cos(e> u r) _ « 2 sin(<» u r) - a x cos(a> u r) + 1 

Vl ~ 2 ' ^2 _ 2 ' 

a 1 =X i co*-2A 2 ol(l + 2C 2 ) + l, a 2 =4f/U; u (l-/l 2 ^) , 

a u _ 2X 2 {l + 2C 2 )-r 1 /2 + f ll T-r l2 _ 4q 3 +r 3 /6 - m r 2 / 2 + ^t ^ 

A(ia + r-Vi) 4^1 + r-^ 3 ity + T-^ 
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are presented here, to make possible to repeat the results obtained by the PID optimization 
from (Matausek & Sekara, 2011). 

Tuning formulae tunl, for integrating processes for M s =2 and m n =2, given by 



0.5904 -0.2707 0.3029 
0.1534 -0.0826 0.0409 
1.2019 -1.5227 1.0714 



-0.1554 0.0311 
-0.0164 0.0033 
-0.4944 0.0916 



and tun2, for processes with the oscillatory dynamics for M s =40 and m n =2, given by 



-8.9189 


63.0913 


0.6494 


-135.2567 


0.2806 


2.2218 


-16.5791 


0.1361 


37.5733 


0.0136 


14.8966 


-82.7969 


-9.0810 


145.2467 


0.9056 



are defined in (Sekara & Matausek, 2011a). The angle (p is in radians. 
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1.4352 
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0.1971 


0.3679 
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0.2371 


0.2291 


4.2403 


0.2291 
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0.9716 


Gp7 


0.8625 


0.1333 


4.1446 


0.3173 


0.3211 


2.3793 


0.5526 


Gp8 


3.8069 


1.8366 


0.6271 


1.4545 


1.6054 


0.7920 


1.1517 


Gp9 


0.6341 


0.5828 


0.9105 


0.9621 


1.0000 


1.6509 


0.5333 



Table 6. Parameters of models G m j(s) of processes G pi (s), j=l,2,...,9. 

Normalized parameters of the PID n controller can be obtained by interpolation based on the 
three points in the p-cp look-up tables of the memorized parameters k n (pi,(pj), km(pi,cpj) and 
kdn(pi,(pj), !=l,2,...,Im, j-l,2,...,]m, determined in advance. In the present paper the look-up 
tables from (Sekara & Matausek, 2011a, Tables 1-4) are used. The four points mash in the p-cp 
look-up tables is presented in Fig. 13. The normalized parameters of the PID n controller, for 
the lower triangle are given by: 



-(l-a-fi)k n21 +ak n 



■PKx 



k in = (1 - « - P)Kl,\ + « fc in2,2 + Ainl, 



= (!-«- fi)h n2 ,i + ak dn22 + pk A 
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where a-au, fi=Pu- The normalized parameters of the PID n controller, for the upper triangle 
are given by: 

K = (1 - « - P)K\,2 + « fc nl,l + fiKl.2 > fc in = (1 - « - /%nl,2 + « fc i„l,l + A„2,2 / 

k dn = (1 " « - /%lnl,2 + « fc dnl,l + /? fc dn2,2 / 

where a=a m , P -pm- In both cases T fn = /c rfn / m n . Then, parameters of the PID controller are 
obtained from (27). 




Fig. 13. The four point mash in the p-tp plane in (Sekara & Matausek, 2011a, Tables 1-4). For 
lower triangle au=(p est -pi,i)/ (pi,2 -pz,i) and pi=((p2,i- <p<xt )/ ( <p 2 ,i - (pi,i). For the upper triangle 
aru=(pi,2-pest )/ (pi,2-pi,i) j3m=(<pest - <pi^)/ ( <p2,2 - <pi,i)- All angles are in degrees and <pi,i< cp 2 ,i, 
cpi,i<cp est < q>2,i, pi,iS p est < p2,l. 
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1. Introduction 

The evolution of process control techniques have increased in a significant way during 
last years. Even so, in the industry, the Proportional-Integral-Derivative (PID) controller is 
frequently used in closed loops due to its simplicity, applicability, and easy implementation 
(Astrom & Hagglund, 1995); Shinskey 1998; (Desbourough & Miller, 2002). An extensive 
research concerning regulatory control of loops used in refinery, chemical, and pulp and 
paper processes reveals that 97% of the applications make use of classical PID structure even 
though sophisticated control techniques, like advanced control strategies, are also based on 
PID algorithms with lower hierarchy level (Desbourough & Miller, 2002). 

Traditionally, the controllers tuning is obtained using classical methods, such as 
Ziegler-Nichols (ZN), Cohen-Coon (CC) and hybridization. However, these methodologies 
are found to present quite satisfactory results for first-order processes, but they usually fail to 
provide acceptable performance for higher-order processes and especially for nonlinear ones 
due to large overshoots and poor regulation on loading (Hang et al., 1991; Mudi et al., 2008). 
In addition, it has been quite difficult to tune properly the PID parameters, during typical 
operation plant, due to difficulties related to production goals (Coelho & Pessoa, 2011). 

Recently, optimization methods through use of information about real or synthetic data, has 
been used as alternative to controllers tuning (Lobato & Souza, 2008). Among these strategies, 
one can cite the based on evolutionary optimization techniques to controllers tuning, such as 
fuzzy logic (Hamid et al., 2010), genetic algorithms (Bandyopadhyay et al., 2001; Pan et al., 
2011), augmented Lagrangian particle swarm optimization algorithm (Sedlaczek & Eberhard, 
2006), particle swarm optimization (Kim et al., 2008; Solihin et al., 2011); differential evolution 
algorithm (Lobato & Souza, 2008); and differential evolution combined with chaotic Zaslavskii 
map (Coelho & Pessoa, 2011). Basically, the interest in evolutionary approach is due to 
following characteristics: easy code building and implementation, no usage of information 
about gradients and, capacity to escape from local optimal (Lobato & Souza, 2008; Souza, 
2007). 
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According to this search area, biological systems have contributed significantly to the 
development of new optimization techniques. These methodologies - known as Bio-inspired 
Optimization Methods (BiOM ) - are based on usage of strategies that seek to mimic 
the behavior observed in species of nature to update a population of candidates to solve 
optimization problems (Lobato et al., 2010). These systems have the capacity to notice and to 
modify its "atmosphere" in order to seek diversity and convergence. In addition, this capacity 
turns possible the communication among the agents (individuals of population) that capture 
the changes in "atmosphere" generated by local interactions (Parrich et al., 2002). 

Among the most recent bio-inspired strategies, one can cite the Bees Colony Algorithm - BCA 
(Pham et al., 2006), the Fish Swarm Algorithm - FSA (Li et al., 2002) and the Firefly Colony 
Algorithm - FCA (Yang, 2008). The classical form of BCA is based on the behavior of bees' 
colonies in their search of raw materials for honey production. In each hive, groups of bees 
(called scouts) are recruited to explore new areas in search for pollen and nectar. These bees, 
returning to the hive, share the acquired information so that new bees are indicated to explore 
the best regions visited in an amount proportional to the previously passed assessment. 
Thus, the most promising regions are best explored and eventually the worst ones end up 
being discarded. This cycle repeats itself, with new areas being visited by scouts at each 
iteration (Pham et al., 2006). The FSA is a random search algorithm based on the behaviour 
of fish swarm which contains searching, swarming and chasing behaviour. It constructs the 
simple behaviours of artificial fish firstly, and then makes the global optimum appear finally 
based on animal individuals' local searching behaviours (Li et al., 2002). Finally, the FCA is 
inspired in social behaviour of fireflies and their communication through the phenomenon 
of bioluminescence. This optimization technique admits that the solution of an optimization 
problem can be perceived as an agent (firefly) which "glows" proportionally to its quality in a 
considered problem setting. Consequently each brighter firefly attracts its partners (regardless 
of their sex), which makes the search space being explored more efficiently (Yang, 2008). 

In the present contribution, BiOM are used for the controllers tuning in chemical engineering 
problems. For this finality, three problems are studied, with emphasis on a realistic 
application: the control design of heat exchangers on pilot scale. The results obtained with 
the methodology proposed are compared with those from the classical methods. This chapter 
is organized as follows. Classical methods to controllers tuning are reviewed in Section 2. In 
Section 3 the main characteristics of BiOM are briefly presented. The results and discussion 
are described in Section 4. Finally, the conclusions and suggestions for future work complete 
the chapter. 

2. Controllers tuning using classical methods 

As mentioned earlier, about 97% of industrial controllers are of PID type, and implement them 
in practice, or even during maintenance of same, there are several technical adjustments of its 
parameters. In literature, there are several classical methods for controllers tuning, such as 
strategies based on minimization of integral error, and correlation-based methods such as ZN 
and CC, among others. 

The majority of works involving the controllers design use the ZN and CC methods (Conner 
& Seborg, 2005; Lobato & Souza, 2008; Solihin et al., 2011; Xi et al, 2007). In this context, the 
ZN and CC methods are brief described. 
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2.1 Reaction curve method 

The principle of this method is the correlation between controller parameters (K c , Tj e Trj) with 
model parameters (K, x and 6) through the temporal response of open-loop system (called the 
process reaction curve ), compared to a step input. In open loop, leads to a unit step of input 
variable to obtain the reaction curve as in Fig. 1. With the parameters 9 and T, we can obtain 




0.632 



f 1 = 9 t 2 =Q + x t 

Fig. 1. Time response to open-loop system with step input y(t). 
the controller parameters according to Tab. 1. 
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Table 1. Controllers tuning with ZN and CC methods through reaction curve (Seborg et al., 
1989). 



2.2 Continuous cycling method 

This classical method is based on sustained oscillation, known as Continuous Cycling Method 
(Seborg et al., 1989). This procedure is valid only for open-loop stable plants, and conducted 
with the following steps: (i) establishment of parameter proportional to a very small gain, (n) 
increase the gain to obtain an oscillatory response with constant amplitude and period (Hi) 
registration of critical value (K u ), critical period (P u ) and (iv) adjustment of the parameters, as 
shown in Tab. 2. Although the vast majority of PID controllers design is tuned by ZN and 
CC methods, some difficulties can be observed, such as the need for knowledge of process 
dynamics in open-loop, and in the Continuous Cycling method, the need to work near of 
system instability limit (Seborg et al., 1989). 
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Table 2. Controllers tuning with Continuous Cycling (Seborg et al., 1989). 
3. Bio-inspired optimization methods 

In the last decades, nature has inspired the development of various optimization methods. 
These techniques try to imitate behaviors of species found in nature, such as ants, birds, 
bees, fireflies, bacteria, among others, to extract information that can be used to promote the 
development of simple and robust strategies. 



the Bee Colony 



This section presents briefly three bio-inspired algorithms in nature: 
Algorithm, the Firefly Colony Algorithm and the Fish Swarm Algorithm. 

3.1 Bee colony algorithm - BCA 

The algorithm proposed by Pham et al. (2006) and described in this section is based on the 
following characteristics observed in nature (von Frisch, 1976): (z) a bees' colony can extend 
itself over long distances (more than 10 km) and in multiple directions simultaneously to 
exploit a large number of food sources, and (ii) capacity of memorization, learning and 
transmission of information in colony, so forming the swarm intelligence. 

In a colony the foraging process begins by scout bees being sent to search randomly for 
promising flower patches. When they return to the hive, those scout bees that found a 
patch which is rated above a certain quality threshold (measured as a combination of some 
constituents, such as sugar content) deposit their nectar or pollen and go to the "waggle 
dance". 

This dance is responsible by the transmission (colony communication) of information 
regarding a flower patch: the direction in which it will be found, its distance from the hive 
and its quality rating (or fitness) (von Frisch, 1976). This dance enables the colony to evaluate 
the relative merit of different patches according to both the quality of the food they provide 
and the amount of energy needed to harvest it (Camazine et al., 2003). Mathematically this 
dance can be represented by following expression: 



x = x — ngh + 2ngh x rand 



(1) 



where x is the new position, ngh is the patch radius for neighbourhood search and rand is the 
random generator. 

After waggle dancing on the dance floor, the dancer (scout bee) goes back to the flower patch 
with follower bees that were waiting inside the hive. More follower bees are sent to more 
promising patches. This allows the colony to gather food quickly and efficiently. While 
harvesting from a patch, the bees monitor its food level. This is necessary to decide upon 
the next waggle dance when they return to the hive (Camazine et al., 2003). If the patch is still 
good enough as a food source, then it will be advertised in the waggle dance and more bees 
will be recruited to that source. 

In this context, Pham et al. (2006) proposed an optimization algorithm inspired by the natural 
foraging behavior of honey bees and presented in Tab. 3. 
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1. Initialise population with random solutions. 

2. Evaluate fitness of the population. 

3. While (stopping criterion not met) 

4. Select sites for neighborhood search. 

5. Recruit bees for selected sites (more bees for the best e sites) and evaluate fitnesses. 

6. Select the fittest bee from each site. 

7. Assign remaining bees to search randomly and evaluate their fitnesses. 

8. End While. 



Table 3. Bees Colony Algorithm (Pham et al., 2006). 

The BCA requires a number of parameters to be set, namely, the number of scout bees 
(n), number of sites selected for neighborhood search (out of n visited sites) (in), number 
of top-rated (elite) sites among m selected sites (e), number of bees recruited for the best e 
sites (nep), number of bees recruited for the other (m-e) selected sites (ngh), and the stopping 
criterion. 

The BCA starts with the n scout bees being placed randomly in the search space. The fitnesses 
of the sites visited by the scout bees are evaluated in step 2. 

In step 4, bees that have the highest fitnesses are chosen as "selected bees" and sites visited 
by them are chosen for neighborhood search. Then, in steps 5 and 6, the algorithm conducts 
searches in the neighborhood of the selected sites, assigning more bees to search near to the 
best e sites. The bees can be chosen directly according to the fitnesses associated with the sites 
they are visiting. 

Alternatively, the fitness values are used to determine the probability of the bees being 
selected. Searches in the neighborhood of the best e sites, which represent more promising 
solutions, are made more detailed by recruiting more bees to follow them than the other 
selected bees. Together with scouting, this differential recruitment is a key operation of the 
BCA. 

However, in step 6, for each patch only the bee with the highest fitness will be selected to 
form the next bee population. In nature, there is no such a restriction. This restriction is 
introduced here to reduce the number of points to be explored. In step 7, the remaining bees 
in the population are assigned randomly around the search space scouting for new potential 
solutions. 

In the literature, various applications using this bio-inspired approach can be found, such 
as: modeling combinatorial optimization transportation engineering problems (Lucie & 
Teodorovic, 2001), engineering system design (Lobato et al., 2010; Yang, 2005), transport 
problems (Teodorovic & Dell'Orco, 2005), mathematical function optimization (Pham et al., 
2006), dynamic optimization (Chang, 2006), optimal control problems (Afshar et al., 2001), 
parameter estimation in control problems (Azeem & Saad, 2004), estimation of radiative 
properties in a one-dimensional participating medium (Ribeiro Neto et al., 2011), among other 
applications (http://www.bees-algorithm.com/). 

3.2 Firefly colony algorithm - FCA 

The FCA is based on the characteristics of fireflies' bioluminescence, insects notorious for 
their light emission. Although biology does not have a complete knowledge to determine all 
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the utilities that firefly luminescence can bring to, at least three functions have been identified 
(Lukasik & Zak, 2009; Yang, 2008): (/) as a communication tool and appeal to potential partners 
in the reproduction, (it) as a bait to lure prey for the firefly, (Hi) as a warning mechanism for 
potential predators reminding them that fireflies have a bitter taste. 

It were idealized some of the flashing characteristics of the fireflies so as to develop 
firefly-inspired algorithms. The following three idealized rules were used (Yang, 2008): 

• all fireflies are unisex so that one firefly will be attracted to other fireflies regardless of their 
sex; 

• attractiveness is proportional to their brightness, thus for any two flashing fireflies the 
less bright will move towards the brightest one. The attractiveness is proportional to the 
brightness and they both decrease as their distance increases. If there is no brightest one, 
than a particular firefly will move randomly; 

• the brightness of a firefly is affected or determined by the landscape of the objective 
function. For a maximization problem, the brightness can simply be proportional to the 
value of the objective function. 

According to Yang (2008), in the firefly algorithm there are two important issues: the variation 
of light intensity and the formulation of the attractiveness. For simplicity, it is always assumed 
that the attractiveness of a firefly is determined by its brightness, which in turn is associated 
with the encoded objective function. 

This swarm intelligence optimization technique is based on the assumption that the solution 
of an optimization problem can be perceived as agent (firefly) which "glows" proportionally 
to its quality in a considered problem setting. Consequently, each brighter firefly attracts its 
partners (regardless of their sex) which make the search space being explored more efficiently. 

The algorithm makes use of a synergic local search. Each member of the swarm explores 
the problem space taking into account results obtained by others, still applying its own 
randomized moves as well. The influence of other solutions is controlled by the value of 
attractiveness (Lukasik & Zak, 2009). 

According to Lukasik & Zak (2009), the FA is presented as follows. Consider a continuous 
constrained optimization problem where the task is to minimize the cost function f(x). 
Assume that there exists a swarm of N agents (fireflies) solving the above mentioned problem 
iteratively and X; represents a solution for a firefly i at the algorithm's iteration k, whereas 
f{Xj) denotes its cost. Initially, all fireflies are dislocated in S (randomly or employing some 
deterministic strategy). Each firefly has its distinctive attractiveness /3 which implies how 
strong it attracts other members of the swarm. As the firefly attractiveness, one should select 
any monotonically decreasing function of the distance r,=d(x,-,x,) to the chosen firefly j, e.g., 
the exponential function: 

jS = |Soe-'> T > (2) 

where /3q and 7 are the following predetermined algorithm parameters: maximum 
attractiveness value and absorption coefficient, respectively. Furthermore, every member of 
the swarm is characterized by its light intensity, I;, which can be directly expressed as the 
inverse of a cost function /(x,). To effectively explore the considered search space S, it is 
assumed that each firefly i changes its position iteratively by taking into account two factors: 
attractiveness of other swarm members with higher light intensity, e.g., L > 7,, V/=l, ..., m, 
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j 7^ i, which is varying across the distance and a fixed random step vector w,. It should be 
noted as well that if no brighter firefly can be found only such randomized step is being used. 

Thus, moving at a given time step t of a firefly i toward a better firefly j is defined as: 

x'r 1 - x l r x j + a (rand - | J (3) 

where the second term on the right hand side of the equation inserts the attractiveness 
factor, /3 while the third term (governed by the parameter a) governs the insertion of certain 
randomness in the path followed by the firefly, rand is a random number between and 1. 

In the literature, few works using the FCA can be found. In this context, the application of the 
technique is emphasized in continuous constrained optimization task (Lukasik & Zak, 2009), 
multimodal optimization (Yang, 2009), solution of singular optimal control problems (Pfeifer 
& Lobato, 2010) and load dispatch problem (Apostolopoulos & Vlachos, 2011). 

3.3 Fish swarm algorithm - FSA 

In the development of FSA, based on fish swarm and observed in nature, the following 
characteristics are considered (Madeiro, 2010): (i) each fish represents a candidate solution 
of optimization problem; (ii) food density is related to an objective function to be optimized 
(in an optimization problem, the amount of food in a region is inversely proportional to value 
of objective function); and (Hi) the aquarium is the design space where the fish can be found. 

As noted earlier, the fish weight at the swarm represents the accumulation of food (e.g., the 
objective function) received during the evolutionary process. In this case, the weight is an 
indicator of success (Madeiro, 2010). 

Basically, the FSA presents four operators classified into two class: "food search" and 
"movement". Details on each of these operators are shown as follows. 

3.3.1 Individual movement operator 

This operator contributes for the movement individual and collective of fishes in swarm. Each 
fish updates its new position using the Eq. (4): 

Xi (t + 1) = x; (t) + rand x s ind (4) 

where x, is the final position of fish i at current generation, rand is a random generator and 
s ind is a weighted parameter. 

3.3.2 Food operator 

The weight of each fish is a metaphor used to measure the success of food search. The higher 
the weight of a fish, the more likely this fish be in a potentially interesting region in design 
space. 

According to Madeiro (2010), the amount of food that a fish eats depends on improvement 
in its objective function in current generation and the value of greatest value considering the 
swarm. The weight is updated according to Eq. (5): 

W '( f + 1 ) = W <W + maxW) (5) 
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where W,(f) is the fish weight i at generation t and A/; is the difference of objective function 
between the current position and the new position of fish i. It is important to emphasize that 
A/,-=0 for the fishes in same position. 

3.3.3 Instinctive collective movement operator 

This operator is important for the individual movement of fishes when A/, 7^0. Thus, only 
the fishes whose individual execution of the movement resulted in improvement of their 
fitness will influence the direction of motion of the school, resulting in instinctive collective 
movement. In this case, the resulting direction (1^), calculated using the contribution of the 
directions taken by the fish, and the new position of the ith fish are given by: 

N 

L Ax/A/, 

h (t) = ^ (6) 

EA/i 

xt{t + l)=Xi(t) + I d {t) (7) 

It is important to emphasize that in the application of this operator, the direction chosen by 
a fish that located the largest portion of food to exert the greatest influence on the swarm. 
Therefore, the instinctive collective movement operator tends to guide the swarm in the 
direction of motion chosen by fish who found the largest portion of food in it individual 
movement. 

3.3.4 Non-Instinctive collective movement operator 

As noted earlier, the fish weight is a good indication of search success for food. In this way, the 
swarm weight is increasing, this means that the search process is successful. So, the "radius" 
of the swarm must decrease for that other regions can be explored. Otherwise, if the swarm 
weight remains constant, the radius should increase to allow the exploration of new regions. 

For the swarm contraction, the centroid concept is used. This is obtained by means of an 
average position of all fish weighted with the respective fish weights, according to Eq. (8): 

N 

E mi (0 
B (0 = ^ (8) 

E Wt (t) 

1=1 

If the swarm weight remains constant in the current iteration, all fish must update their 
positions using the Eq. (9): 

xlt)-B(t) 

x (t + i) = x{t )-^x nH ix I ±L^L (9) 

where d is a function that calculates the Euclidean distance between the centroid and the 
current position of fish, and s vo i is the step size used to control the displacement of fish. 

In the literature, few works using the FSA can be found. In this context, feed forward neural 
networks (Wang et al., 2005), parameter estimation in engineering systems (Li et al, 2004), 



A Comparative Study Using Bio-Inspired Optimization Methods Applied to Controllers Tuning 151 

combinatorial optimization problem (Cai, 2010), global optimization (Yang, 2010), Augmented 
Lagrangian fish swarm based method for global optimization (Rocha et al., 2011), forecasting 
stock indices using radial basis function neural networks optimized (Shen et al., 2011), and 
hybridization of the FSA with the Particle Swarm Algorithm to solve engineering systems 
(Tsai& Lin, 2011). 

4. Applications 

For evaluating the methodology proposed in this work for controllers tuning, some practical 
points should be emphasized: 

• the objective function (Sum Quadratic Error - SQE) considered in all case studies is given 

by Eq. (10): 

np np 2 

min SQE = £ Error = £ (x set P omt - x calculated ) (10) 

k=l k=l 

where X set P mnt and x are the values of variables considered at setpoint and 

calculated using the mathematical model, respectively, and np is the points number used 
to formulate this objective function (np equals to 1000). 

• in all case studies the following parameters used are presented in Tab. (Li et al., 2002; Pham 
et al., 2006; Yang, 2008). 

• it should be emphasized that is necessary, with the parameters listed in this table, 1510 
objective function evaluations in each algorithm. 

• all case studies were run 20 times independently to obtain the values and standard 
deviations shown in the upcoming tables. 

• the stopping criterion used was the maximum number of iterations (generations). 

• to compare the results obtained by the BiOM, the following strategies were used: 
Ziegler-Nichols Sensibility-Limiar Method (ZN-SL), Ziegler-Nichols Reaction Curve 
Method (ZN-RC) and Cohen-Coon Reaction Curve Method (CC-RC). 

4.1 Distillation column 

This first study proposed by (Skogestad & Morari, 1987) considers a distillation column of 
high purity consisting of 25 plates, a condenser and a reboiler. The reflow ration and the 
composition of distillate are the input and output system, respectively. The dynamic model 
that describes this system is given by following transfer function (Skogestad & Morari, 1987): 

„, , -0.75448z + 0.149199 

G(2) = z 0.6386913 (11) 

The objective is to maintain the composition of distillate in 0.99 by manipulating the reflow 
ratio, which has a nominal value of 1.477 Kmol/min. In this case, the following ranges to 
controllers tuning are considered: < K c < 150, < Tj < 50 and < Tq < 50. 

Table 5 presents the best value and standard deviation for the distillation column case study. 

In this table can be observed that both the algorithms presented good estimates for the 
unknown parameters. When the results are analyzed in terms of the objective function (OF), 
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BCA 


Number of scout bees 


10 


Number of bees recruited for the best e sites 


5 


Number of bees recruited for the other selected sites 


5 


Number of sites selected for neighbourhood search 


5 


Number of top-rated (elite) sites among m selected sites 


5 


Neighborhood search 


10~ 6 


Generation Number 


50 


FCA 


Number of fireflies 


15 


Maximum attractiveness value 


0.9 


Absorption coefficient 


0.9 


Generation Number 


50 


FSA 


Number of fishes 


15 


Weighted parameter (s,„^) 


0.01 


Weighted parameter (s TO ;) 


1 


Generation Number 


50 



Table 4. Parameters used by the BiOM. 



Method 


K c 


Tj (min l ) 


t d (min x ) 


OF (Eq. 10) 


ZN-SL 


67.2000 


12.500 


3.1250 


8.10xlO- J 


ZN-RC 


2.6578 


2.0000 


0.5000 


1.24 xl0~ 2 


CC-RC 


3.2890 


2.1154 


0.3364 


1.09xl0~ 2 


BCA 


24.282 
(36.265) 


0.008 
(12.12) 


43.103 
(12.710) 


8.102x10^ 
(3.026xKT 6 ) 


FCA 


77.412 
(37.324) 


0.003 
(0.037) 


26.955 
(15.926) 


8.100xlO~ J 
(6.834xl0~ 8 ) 


FSA 


128.009 
(28.260) 


0.009 
(0.237) 


7.176 
(18.779) 


8.101 xl0- J 
(3.145 xl0~ 8 ) 



Table 5. Results obtained by the BiOM - distillation column case study. 

is clear that the combination of control parameters lead us to very close values, also seen in 
the value of standard deviation presented. 

Figure 2 present the distillation top and the control action (reflow profile), respectively, using 
the classical methods and the BiOM. The behaviour observed in this simple case study is 
practically the same for all strategies used. 



4.2 Heat exchanger 

Consider a heat exchanger type shell-tube counter-current as illustrated in Fig. 3 (Garcia, 
2005). In this figure, Q c j and T c ; represent, the flow rate and inlet temperature of the hot 
fluid, respectively, Qf /£ . and Tf iL ,, the flow rate and inlet temperature of cold fluid, respectively. 
T c is the fluid temperature on the side of hull and Tj is the fluid temperature in the side of 
pipe. The objective of this system is to heat a water stream at 40 °C to 41 °C manipulating 
a hot water stream (Qt, e ) with nominal flow rate 0.0004 m 3 /s. The thermal exchanges are 
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Fig. 2. Distillation top profile (a) and reflow profile (b) using classical and BiOM. 

considered: heat transfer fluid circulating between the tubes and the hull, heat transfer fluid 
circulating between the hull and its walls, and transport of energy (enthalpy) due to fluid flow 
in pipes and shell. More information about the design and the considerations are in Garcia 
(2005). 

T Q 

c,s , ^c,s 



-A 



L 



% 



J 



T 



% 



T Q 

c,e ^-c.e 
Fig. 3. Schematic heat exchanger. 

The dynamic model that describes this system is given by transfer function: 

0.0189 



G(s) 



10s 3 + 5.114s 2 + 0.825s + 0.041 



(12) 



In this case, the following ranges to controllers tuning are considered: < K c < 150, < Tj < 
50 and < D < 50. 

Table 6 presents the best value and standard deviation for the heat exchanger case study. 

As observed in the previous case, that both the algorithms presented good estimates for the 
unknown parameters, but the best results were obtained by the BiOM. It is possible to observe 
the fluctuation of control parameters, found in the value of standard deviation. 

Figure 4 present the temperature and flow profiles using the classical methods and the BiOM. 
It should be emphasized the oscillatory behaviour observed with the application of the BiOM, 
even for a short period of time (see Fig. 4(a)). 
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Method 


K c 


Tj (s" 1 ) 


t D (s- 1 ) 


OF (Eq. 10) 


ZN-SL 


5.8800 


30.0000 


7.5000 


0.6206 


ZN-RC 


3.3600 


27.0000 


6.7500 


2.1733 


CC-RC 


4.4300 


26.0200 


4.2500 


0.9025 


BCA 


26.289 
(10.949) 


10.086 
(5.845) 


17.120 
(4.467) 


0.0219 

(0.0202) 


FCA 


48.638 
(13.425) 


6.577 
(5.146) 


24.354 
(4.898) 


0.0228 
(0.033) 


FSA 


47.274 
(9.751) 


9.012 
(7.209) 


17.508 
(5.409) 


0.0238 
(0.0157) 



Table 6. Results obtained by the BiOM - heat exchanger case study. 
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Fig. 4. Temperature and flow (control action) profiles. 

4.3 Shell and tube heat exchanger 

Finally, consider the real system shown in Fig. 5 for analysis and application of the previously 
studied concepts. This system consists essentially of: (/') the main tank (P5) in stainless steel 
with a capacity of approximately 0.250 m , (it) stainless steel shell and tube heat exchanger 
(PI), (Hi) positive displacement pump for movement of food products (P3), (iv) centrifugal 
pump for the heating agent movement (P2) and (v) vertical cylindrical storage tank water 
heater (P4) (Gedraite et al., 2011). 

The Vettore-Manghi heat exchanger is responsible for heating the liquid food product that 
flows inside the tube bundle, considering four passes. The displacement of the process fluid 
inside the tubes of the heat exchanger is driven by a model RE50-110 Robuschi positive 
displacement pump (pump 2). The heating of the heat exchanger is done by hot water which 
flows through the shell side of the exchanger. Hot water is transported by Robuschi , model 
RE50-160 centrifugal pump (pump 3). Hot water is heated at the expense of saturated steam 
produced in the H. BREMER steam generator, installed in suitable and safe environment. The 
temperature of the process fluid is controlled by manipulating the flow of steam fed to the 
system, whose setting is done by the Fluxotrol model PK2117 control valve (P4), with reverse 
action. The hot water removed from the shell of the heat exchanger returns to the vertical tank, 
that is equipped with a safety valve. For cooling the product, the procedure is reversed, ie, the 
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Fig. 5. Process and instrumentation diagram for the system studied (Gedraite et al., 2011). 

flow control valve used for manipulate the value of heating steam flow is gradually closed. 
In this process, the response time is slower when compared with heating time. Cooling only 
occurs as a result of the heat exchange between the body of the heat exchanger, the process 
fluid and the environment. 

The data acquisition and control system is composed by an PC based DAS (Data Acquisition 
System), working also as a computer control. This system consists of the following items: (i) 
PC microcomputer for the collection and storage of process data, (ii) Lab VIEW® version 2009 
application to perform monitoring, data acquisition and process control in real-time, (Hi) data 
acquisition board, National Instruments (NI) PCI-6259 model, with 4 analog output channels 
and 32 analog input channels with both operating range of -10 V to +10 V and resolution of 
16 bits, and 48 channel digital input/output programmable, (iv) set of cables to acquisition 
board NI model SHC68-68-EPM, (v) a connections terminal NI model CB-68LP, (vi) signal 
conditioners INCON model CS01-1360 to match the signals from the sensing elements of 
temperature, (vii) temperature sensors IOPE model 49312 type Pt 100, (viii) METROVAL flow 
meter model OI-2-SMRX/FS, (ix) ENGINSTREL model 621-IPB electrical current to pressure 
signal converter, (x) pneumatic control valve Fluxotrol model PK2117 and (x,) Micronal model 
B474 pH meter. 



4.3.1 Approximate model system 

The non-parametric identification process employs basically the response curves of the system 
when excited by input signals like step, impulse or sinusoidal. From these curves, one can 
extract approximate models of low order, which describe the dynamic behavior of the process 
(Aguirre, 2007). These models are reasonably accurate and can be assumed to be good enough 
to represent the system studied. In this work, they were used to perform the pre-tuning PID 
controllers and to mathematically model the dynamic behavior of pH versus time. 

The input most commonly used as non-parametric excitation to identify a process dynamics 
is the step (Aguirre, 2007). These tests usually can generate by means of graphical 
representation, empirical dynamic models that consists of low order transfer functions (1st 
or 2nd order, possibly including a dead time) with a maximum of four parameters to be 
determined experimentally. 
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Astrom & Hagglund (1995) state that many of the processes can be represented in an 

approximate way, by combining four elements typically found in industrial processes, 

namely: (i) gain, (n) transport delay, {Hi) transfer delay and (iv) integrating element. The 

approach of overdamped systems of order 2 or higher for transfer delay plus dead time 

(transport delay) can be represented by the transfer function shown in Eq. (13) (Aguirre, 

2007): 

Ke~ 6s 
G (s) = —— (13) 



where K is the gain, T is the transfer delay and 9 is the dead time (or transport delay). 



4.4 Plant reaction curves 

Tests were made to obtain the process parameters related to plant response to changes in flow 
and temperature. In this test, the equipment was put into operation with steady flow of 7 
Lmin -1 and applied positive step 3 Lmin -1 at time 32 s, waiting for the system stabilization. 
In the sequence, a negative step of 3 Lmin at time 203 s was applied. The first step (7 to 10 
Lmin ) was adopted to obtain the process parameters, whose results are presented below. 
Figure 6 shows the system behavior to the situation examined. 
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Fig. 6. Step test at flow of process fluid. 

In the assay realized for the temperature, whose response time is illustrated in Fig. 7, a 
constant flow of 9 Lmin -1 was used. The outlet temperature of process fluid was adjusted 
equal to 60 °C and a step into the control valve installed at the steam line stem position 
was applied at the time 50 s, starting from the condition of fully closed until 50% opening. 
Following the instant 1430 s, we applied a second step of amplitude equal to 10%. The analysis 
to obtain the process parameters were calculated considering the first step (50%). 

The process parameters K (process gain), T (process time constant) and 6 (process dead 
time) were calculated using the method proposed by Aguirre (2007). The transfer functions 
obtained are presented in Eqs. (14) and (15): 

i) flow: K=1.3 Lmin^V- 1 , t=14 s and 6=2 s 



G(s) 



1.3exp(-2s) 
l + 14s 



(14) 
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Fig. 7. Step test at temperature position at steam line. 
ii) temperature: £=7.22 °C, t=378 s and 6=78 s 

7.22 exp(- 78s) 



G(s) 



1 + 378s 



(15) 



In these simulations, the following ranges to design parameters are considered: < K c < 50, 
< T 7 < 248 and < t d < 50. 

Tables 7 and 8 present the average and standard deviation for the flow and the temperature 
case studies. 



Method 


K c 


Tj (s- 1 ) 


td (s- 1 ) 


OF (Eq. 10) 


ZN-SL 


6.9240 


3.25 


0.8125 


0.0119 


ZN-RC 


3.1185 


8.0000 


2.0000 


0.0629 


CC-RC 


3.7162 


8.9937 


1.3972 


0.0366 


BCA 


9.4859 
(1.6760) 


3.7011 
(0.5122) 


0.6687 
(0.1531) 


1.0707xl0- b 
(0.0055) 


FCA 


9.1635 
(1.2686) 


3.9305 
(0.7119) 


0.7154 
(0.1616) 


2.2288 xlO" 7 
(0.0021) 


FSA 


9.3345 
(0.8472) 


3.8780 
(0.7640) 


0.7149 
(0.2455) 


3.3536 xlO" 7 
(0.0001) 



Table 7. Results obtained by BiOM - Flow case study. 

In these tables is possible to observe that both the algorithms presented good estimates for the 
controllers tuning, but the best results were obtained by the BiOM (this represent a reduction 
of approximately 97% in comparison to ZN-SL method). In addition, it is important to 
comment that if a larger range for the design variables was used, the value of the objective 
function would reduce. However, in spite of this reduction, the design found cannot be 
physically viable, e.g., can represent an infeasible condition in industrial context, as illustrated 
in Fig. 11(a) for the classical methods. 

Figures 8 and 9 present the flow and temperature profiles using the classical methods and the 
BiOM. Also can be observed in these figures the control action (motor pump signal (8(a)) and 
valve steam signal (9(b)). 
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Method 


K c 


Tj (s" 1 ) 


r D (s- 1 ) 


OF (Eq. 10) 


ZN-SL 


0.8880 


112.5 


28.125 


87303 


ZN-RC 


0.7453 


124.0000 


31.0000 


64056 


CC-RC 


0.8758 


142.8046 


21.9605 


77922 


BCA 


1.3713 
(0.0010) 


248.0000 
(0) 


38.1768 
(0.0307) 


1929.1287 
(0.2111) 


FCA 


1.3708 
(0.0018) 


248.0000 
(0.0008) 


38.1905 
(0.0518) 


1929.1073 
(0.1990) 


FSA 


1.3725 
(0.0015) 


248.0000 
(0.0023) 


38.1428 
(0.0438) 


1929.1481 
(0.1501) 



Table 8. Results obtained by BiOM - Temperature case study. 
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Fig. 8. Flow profile and control action (motor pump signal). 
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Fig. 9. Temperature profile and control action (valve steam signal). 
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5. Conclusions 

In the present contribution, the effectiveness of using the BiOM for controllers tuning through 
formulation of an optimization problem was analyzed. 

In this sense, three cases were studied and it was possible to conclude that both bio-inspired 
algorithms led to good results for an acceptable number of generations (1510) when compared 
to the classical methods. It should be pointed out that the quality of solution obtained is 
dependent of design space considered, e.g., if other ranges were used, other results can be 
found. Besides, also can be observed that the combination of control parameters, can take to 
values close, in terms of the objective function. 

It is important to emphasize that the use of the BiOM not have the pretension of substituting 
the classical techniques for the controllers tuning, but to represent an interesting alternative 
for this purpose. 
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1. Introduction 

A coordinated group of robots can execute certain tasks, e.g. surveillance of large areas 
(Hougen et al., 2000), search and rescue (Jennings et al., 1997), and large objects- 
transportation (Stouten and De Graaf, 2004), more efficiently than a single specialized robot 
(Cao et al., 1997). Other tasks are simply not accomplishable by a single mobile robot, 
demanding a group of coordinated robots to perform it, like the problem of sensors and 
actuators positioning (Bicchi et al., 2008), and the entrapment/ escorting mission (Antonelli 
et al., 2008). In such context, the term formation control arises, which can be defined as the 
problem of controlling the relative postures of the robots of a platoon that moves as a single 
structure (Consolini et al., 2007). 

Mobile manipulator is nowadays a widespread term that refers to robots built by a robotic 
arm mounted on a mobile platform. This kind of system, which is usually characterized by a 
high degree of redundancy, combines the manipulability of a fixed-base manipulator with 
the mobility of a wheeled platform. Such systems allow the most usual missions of robotic 
systems which requiere both locomotion and manipulation abilities. Coordinated control of 
multiple mobile manipulators have attracted the attention of many researchers (Khatib et al., 
1996; Fujii et al., 2007; Tanner et al., 2003; Yasuhisa et al., 2003). The interest in such systems 
stems from the capability for carrying out complex and dexterous tasks which cannot be 
simply made using a single robot. Moreover, multiple small mobile manipulators are also 
more appropriate for realizing several tasks in the human environments than a large and 
heavy mobile manipulator from a safety point of view. 

Main coordination schemes for multiple mobile manipulators that can be found in the 
literature are: 

1. Leader-follower control for mobile manipulator, where one or a group of mobile 
manipulators plays the role of a leader, which track a preplanned trajectory, and the 
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rest of the mobile manipulators form the follower group which moves in conjunction 
with the leader mobile manipulators (Fujii et al., 2007; Hirata et al., 2004; Thomas et al., 
2002). In Xin and Yangmin, 2006, a leader-follower type formation control is designed 
for a group of mobile manipulators. To overcome parameter uncertainty in the model of 
the robot, a decentralized control law is applied to individual robots, in which an 
adaptive NN is used to model robot dynamics online. 
2. Hybrid position-force control by decentralized/ centralized scheme, where the position 
of the object is controlled in a certain direction of the workspace and the internal force 
of the object is controlled in a small range of the origin (Khatib et al., 1996; Tanner et al., 
2003; Yamamoto et al., 2004). In Zhijun et al., 2008, robust adaptive controllers of 
multiple mobile manipulators carrying a common object in a cooperative manner have 
been investigated with unknown inertia parameters and disturbances. At first, a concise 
dynamics consisting of the dynamics of mobile manipulators and the geometrical 
constraints between the end-effectors and the object is developed for coordinated 
multiple mobile manipulators. In Zhijun et al., 2009 coupled dynamics are presented for 
two cooperating mobile manipulators manipulating an object with relative motion in 
the presence of uncertainties and external disturbances. Centralized robust adaptive 
controllers are introduced to guarantee the motion and force trajectories of the 
constrained object. A simulation study to the decentralized dynamic control for a robot 
collective consisting of nonholonomic wheeled mobile manipulators is performed in 
Hao and Venkat, 2008, by tracking the trajectories of the load, where two reference 
signals are used for each robot, one for the mobile platform and another for end-effector 
of the manipulating arm. 

To reduce performance degradation, on-line parameter adaptation is relevant in 
applications where the mobile manipulator dynamic parameters may vary, such as load 
transportation. It is also useful when the knowledge of the dynamic parameters is limited. 
As an example, the trajectory tracking task can be severely affected by the change imposed 
to the robot dynamics when it is carrying an object, as shown in (Martins et al., 2008). Hence, 
some formation control architectures already proposed in the literature have considered the 
dynamics of the mobile robots (Zhijun et al., 2008; Zhijun et al., 2009). 

In this Chapter, it is proposed a novel method for centralized-decentralized coordinated 
cooperative control of multiple wheeled mobile manipulators. Also, it is worth noting that, 
differently to the work in Hao and Venkat, 2008, we use a single reference for the end- 
effector of the robot mobile manipulator. 

Although centralized control approaches present intrinsic problems, like the difficulty to 
sustain the communication between the robots and the limited scalability, they have technical 
advantages when applied to control a group of robots with defined geometric formations. 
Therefore, there still exists significant interest in their use. As an example, in Antonelli et al., 
2008, a centralized multi-robot system is proposed for an entrapment/ escorting mission, 
where the escorted agent is kept in the centroid of a polygon of n sides, surrounded by n 
robots positioned in the vertices of the polygon. Another task for which it is important to keep 
a formation during navigation is large-objects transportation, since the load has a fixed 
geometric form. Another recent work dealing with centralized formation control is Mas et al., 
2008, where a control approach based on a virtual structure, called Cluster Space Control, is 
presented. There, the positioning control is carried out considering the centroid of a geometric 
structure corresponding to a three-robot formation. 
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In this Chapter, the proposed strategy conceptualizes the mobile manipulators system (with 
n > 3 ) as a single group, and the desired motions are specified as a function of cluster 
attributes, such as position, orientation, and geometry. These attributes guide the selection 
of a set of independent system state variables suitable for specification, control, and 
monitoring. The control is based on a virtual 3-dimensional structure, where the position 
control (or tracking control) is carried out considering the centroid of the upper side of a 
geometric structure (shaped as a prism) corresponding to a three-mobile manipulators 
formation. It is worth noting that in control problem formulating first it is considered three 
mobile manipulators robots, and then is generalized to mobile manipulators robots. 

The proposed multi-layer control scheme is mainly divided in five modules: 1) the upper 
module is responsible for planning the trajectory to be followed by the team of mobile 
manipulators; 2) the next module controls the formation, whose shape is determined by the 
distance and angle between the end-effector of a mobile manipulator and the two other 
ones; 3) another module is responsible to generate the control signals to the end-effectors of 
the mobile manipulators, through the inverse kinematics of each robot. As a mobile 
manipulator is usually a redundant system, this redundancy can be used for the 
achievement of additional performances. In this layer two secondary objectives are 
considered: the avoidance of obstacles by the mobile platforms and the singular 
configuration prevention through the control of the system's manipulability; introduced by 
Yoshikawa (1985). 4) The adaptive dynamic compensation module compensates the 
dynamics of each mobile manipulator to reduce the velocity tracking error. It is worth 
noting that this controller has been designed based on a dynamic model having reference 
velocities as input signals. Also, it uses a robust updating law, which makes the dynamic 
compensation system robust to parameter variations and guarantees that no parameter drift 
will occur; 5) finally, the robots module represents the mobile manipulators. 

It is worth noting that we propose a methodology to avoid obstacles in the trajectory of any 
mobile manipulator based on the concept of mechanical impedance of the interaction 
robots-environment, without deforming the virtual structure and maintaining its desired 
trajectory. It is considered that the obstacle is placed at a maximum height that it does not 
interfere with the workspace, so that the arm of the mobile manipulators can follow the 
desired trajectory even when the platform is avoiding the obstacle. 

This Chapter is organized as follows. Section 2 shows the kinematic and dynamic models of 
the mobile manipulator. Section 3 presents the proposed multi-layer control scheme for the 
coordinated and cooperative control of mobile manipulators. While the forward and inverse 
kinematics transformations, necessary for the control scheme, are presented in Section 4. 
Section 5 describes the scalability for coordinated cooperative control of mobile 
manipulators. By its turn, Section 6 presents the design of the controller, and the analysis of 
the system's stability is developed. Next, simulation experiments results are presented and 
discussed in Section 7, and finally the Chapter conclusions are given in Section 8. 

2. Mobile manipulator models 

The mobile manipulator configuration is defined by a vector q of n independent 
coordinates, called generalized coordinates of the mobile manipulator, where 
1 = ['?i Hi ••• 1n\ = [l» 1o 1 where q n represents the generalized coordinates of the 
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arm, and q the generalized coordinates of the mobile platform. We notice that n=n a +n , 
where n a and n are respectively the dimensions of the generalized spaces associated to 
the robotic arm and to the mobile platform. The configuration q is an element of the mobile 
manipulator configuration space; denoted by JV . The location of the end-effector of the 
mobile manipulator is given by the m -dimensional vector \\=[h l h 2 ... h m ] which 
define the position and the orientation of the end-effector of the mobile manipulator in X. . 
Its m coordinates are the operational coordinates of the mobile manipulator. The set of all 
locations constitutes the mobile manipulator operational space, denoted by M . 

The location of the mobile manipulator end-effector can be defined in different ways 
according to the task, i.e., it can be considered only the position of the end-effector or both 
its position and its orientation. 

2.1 Mobile manipulator kinematic model 

The kinematic model of a mobile manipulator gives the location of the end-effector h as a 
function of the robotic arm configuration and the platform location (or its operational 
coordinates as functions of the robotic arm generalized coordinates and the mobile platform 
operational coordinates). 

(q a >q P ) i-> h =/(q a 'q P ) 

where, !N ' a is the configuration space of the robotic arm, 3t ' is the operational space of the 
platform. 

The instantaneous kinematic model of a mobile manipulator gives the derivative of its end- 
effector location as a function of the derivatives of both the robotic arm configuration and 
the location of the mobile platform, 

where, h = [h\ h 2 ... h m ] is the vector of the end-effector velocity, v = [v 1 v 2 ... v s ] 

= [v v a ] is the control vector of mobility of the mobile manipulator. Its dimension is 
S n = 8 + S m , where 8 and S m are respectively the dimensions of the control vector of 
mobility associated to the mobile platform and to the robotic arm. Now, after replacing 
J(q) = — (q a ,q )T(q) in the above equation, we obtain 

n(f) = j(q)v(0 (1) 
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where, J(q) is the Jacobian matrix that defines a linear mapping between the vector of the 
mobile manipulator velocities v(t) and the vector of the end-effector velocity h(f), and 
T(q) is the transformation matrix that relates joints velocities q(f) and mobile manipulator 
velocities v(f) such that q(f) = T(q)v(f) . 

Remark 1: The transformation matrix T(q) includes the non-holonomic constraints of the 
mobile platform. 

The Jacobian matrix is, in general, a function of the configuration q ; those configurations at 

which J(q) is rank-deficient are termed singular kinematic configurations. It is fundamental to 

notice that, in general, the dimension of the operational space m is less than the degree of 
mobility of the mobile manipulator, therefore the system is redundant. 

2.2 Mobile manipulator dynamic model 

The mathematic model that represents the dynamics of a mobile manipulator can be 
obtained from Lagrange's dynamic equations, which are based on the difference between 
the kinetic and the potential energy of each of the joints of the robot (energy balance) (Spong 
and Vidyasagar, 1989; Yoshikawa, 1990; Sciavicco and Siciliano, 2000). The dynamic 
equation of the mobile manipulator can be represented as follows, 

M(q)v + C(q,v)v + G(q) + d = B(q)T (2) 

where, q = [q 1 ,,..,q n \ e *R" is the general coordinate system vector of the mobile 
manipulator, v = [v 1 ,....,Vg ] e W " is the velocity vector of the mobile manipulator, 
M(q) e SR " x " is a symmetrical positive definite matrix that represents the system's inertia, 
C(q,v)v s 9? " represents the components of the centripetal and Coriolis forces, 
G(q) e 5? " represents the gravitational forces, d denotes bounded unknown disturbances 

including the unmodeled dynamics, t eSR " is the torque input vector, B(q) g <R " xd » is the 
transformation matrix of the control actions. 

Most of the commercially available robots have low level PID controllers in order to follow 
the reference velocity inputs, thus not allowing controlling the voltages of the motors 
directly. Therefore, it becomes useful to express the dynamic model of the mobile 
manipulator in a more appropriate way, taking the rotational and longitudinal reference 
velocities as the control signals. To do so, the velocity servo controller dynamics are 
included in the model. The dynamic model of the mobile manipulator, having as control 
signals the reference velocities of the system, can be represented as follows, 

M(q)v + C(q,v)v + G(q) + d=v ref (3) 
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where M(q) = H" 1 (M + D) / C(q,v) = H" 1 (C + P) , G(q) = H _1 G(q) and d = H _1 d . Thus, 
M(q) e m g " xg " is a positive definite matrix, C(q,v)v e 9i S " , G(q) e 9T 5 " , d e 3l d " and 

v re£ e s Jt " is the vector of velocity control signals, H e *R " x " , D e SR " x " and P e 9? " x " 

are positive definite constant diagonal matrices containing the physical parameters of the 
mobile manipulator, motors, and velocity controllers of both the mobile platform and the 
manipulator. It is important to remark that H, D and P are positive definite constant 
diagonal matrices, hence the properties for the dynamic model with reference velocities as 
control signals (3) were obtained on based of the properties of the dynamic model (2): 

Property 1. Matrix M(q) is positive definite, additionally it is known that 

|M(q)|<fc M 
Property 2. Furthermore, the following inequalities are also satisfied 

|C(q,v)|<fc c ||v| 
Property 3. Vector G(q) and d are bounded 

|G(q)|<*G ; \\d\\<k d 
where, k c , k M , k G and k d denote some positive constants. 
Property 4. The dynamic model of the mobile manipulator can be represented by 

M(q)v+C(q,v)v + G(q) + d = <D(q,v)x 

where, <t>(q,v) s s Jt" x and X = [Zi Xi ■■■ Zi] is the vector of / unknown parameters of 

the mobile manipulator, i.e., mass of the mobile robot, mass of the robotic arm, physical 
parameters of the mobile manipulator, motors, velocity, etc. 

For the sake of simplicity, from now on it will be written M=M(q), C = C(q,v) and 
G=G(q). 

Hence, the full mathematical model of the mobile manipulator robot is represented by (1), 
the instantaneous kinematic model and (3), the dynamic model, taking the reference velocities of 
the system as input signals. 

3. Multi-layers control scheme 

Figure 1, shows the Multi-layer control Scheme of the coordinated cooperative control of 
mobile manipulators which is taken into account in this Chapter. 

Each layer works as an independent module, dealing with a specific part of the problem of 
coordinated cooperative control, and such control scheme includes a basic structure defined 
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by the formation control layer, the kinematic control layer, the robots layer and the 
environment layer. 
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Fig. 1. Multi-layer control scheme 

• The Off-line Planning layer is responsible for setting up the initial conditions, thus 
generating the trajectory of the object to be tracked, and for establishing the desired 
structure. The On-line Planning layer is capable of changing the references in order to 
make the formation to react to the environment, e.g., to modify the trajectory to avoid 
obstacles (it should be included only when a centralized obstacle avoidance strategy is 
considered; in this work it is considered the decentralized obstacle avoidance). 

• The Formation Control layer is responsible for generating the control signals to be sent to 
the mobile manipulators, working as a team, in order to reach the desired values 
established by the planning layers. 

• The Kinematic Control layer is responsible for generating the control signals to the end- 
effector of the mobile manipulators considering different control objectives. 

• The Adaptive Dynamic Compensation layer compensates the dynamics of each robot to 
reduce the velocity tracking error. 

• The Robot layer represents the mobile manipulators (mobile manipulators with unicycle- 
like mobile platforms, car-like and/ or omnidirectional type), and finally; 

• The Environment layer represents all objects surrounding the mobile manipulators, 
including the mobile manipulators themselves, with their external sensing systems, 
necessary for implementing obstacle avoidance. 
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One of the main advantages of the proposed scheme is the independence of each layer, 
i.e., changes within a layer do not cause structural changes in the other layers. As an 
example, several kinematic controllers or dynamic compensation approaches can be 
tested using the same formation control strategy and vice-versa. It is worth mentioning 
that a simple structure can be obtained from the presented scheme, that is, some layers 
can be eliminated whenever the basic structure is maintained and the absence of the 
eliminated layers do not affect the remaining layers. For example, the On-line Planning 
layer could be discarded in the case of trajectory tracking or path following by a multi- 
robot formation in a known environment free of obstacles, because the entire task 
accomplishment is controlled by the Formation Control layer. Also the Adaptive Dynamic 
Compensation layer can be suppressed, for applications demanding low velocities and 
light load transportation. 

On the other hand, it is important to stress that some additional blocks are necessary to 
complete the multi-layer scheme, such as J F (r) and /(x), which represents the inverse 

formation Jacobian matrix, and the forward kinematic transformation function for the 
formation, respectively. 

Remark 2: The mobile manipulators can be different, i.e., each mobile manipulator can be 
built by different types mobile platforms or/ and different types robotic arms. Thus each 
mobile manipulator has its own configuration. 

Remark 3: A mobile manipulator is defined as a redundant system because it has more 
degrees of freedom than required to achieve the desired end-effector motion. Hence, the 
redundancy of such systems can be effectively used for the achievement of additional 
performances. 

4. Kinematic transformation 

The proposed coordinated cooperative control method considers three or more mobile 
manipulators. In the first step, only three mobile manipulators are considered. In this case 
the control method is based on creating a regular or irregular prism defined by the position 
of the end-effector of each mobile manipulator. The location of the upper side of the prism 
in the plane X-Y of the global framework is defined by P F = [x F y F if F ], where (x F , y F ) 
represents the position of its centroid, and y/ F represents its orientation with respect to the 
global Y-axis. The structure shape of the prism (regular or irregular) is defined by 
Sf = [Pf (Jf Pr z if Z 2F z 3F ], where, p F represents the distance between hj and h 2 , 
Cj F the distance between h x and h 3 , J3 F the angle formed by h 2 hjh 3 and (z 1F , z 2F , z 3F ) 
represents the height of the upper side of the prism. This situation is illustrated in Figure 2. 

Remark 4: h ; represents the position the end-effector of the f-th mobile manipulator. 

The relationship between the prism pose-orientation-shape and the end-effector positions of 
the mobile manipulators is given by the forward and inverse kinematics transformation, i.e., 

r = /(x) and x = / _1 (r) , where r = [P F S F f and x = [h[ h 2 h 3 ]. 
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Fig. 2. Structure variables 
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The forward kinematic transformation /(.) , as shown in figure 2, is given by 
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where, r F = 4{ x 2~ x ?>) + (3/2 — 3/3 ) • ^ n tu 111 / ror the inverse kinematic transformation 
/" (.), two representations are possible, depending on the disposition of the mobile 

manipulators in the prism shape (clockwise or counter-clockwise). Such disposition can be 
referred to as R 1 R 2 R 3 or R 1 R 3 R 2 sequence (R { represents the i-th mobile manipulator 

robot). Considering the first possibility, x = f£ R R (r) is given by, 



x F +^h F sini// F 



y F + jh F costf/ F 



x F +^h F sini// F -p F sin(a +if/ F ) 
y F +jh F cosi// F -p F cos(a + y/ F ) 



x F + |/i F su\i// F + q F sin(/7 F - a - y/ F ) 
y F + §/i F cos t// F -q F cos{p F -a-y/ F ) 

Z 3F 
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where, h F = Jiipf +(Jp ~\ r F ) represents the distance between the end-effector hj and the 
point in the middle of the segment h 2 h 3 , passing through { x t'})f)' anc ^ 

p F +hj -\r F i 

a = arccos - — . On the other hand, x = f R R R? (r) is given by 

2p F h F 



x F + ^h F sint// F 
y F +^h F cosi// F 



x F + ^h F sini// F +p F sin(a - i// F ) 
y F + ^-h F cos<f/ F -p F cos(a -y/ F ) 

x F +^h F sini// F -q F sin[/3 F -a + y/ F ) 
y F +^h F cosi// F ~q F cos(/3 F -a + i// F ) 



Figure 3 shows the control structure proposed in this Chapter for the coordinated 
cooperative control of mobile manipulators. Taking the time derivative of the forward and 
the inverse kinematics transformations we can obtain the relationship between the time 
variations of x(f) and r(f) , represented by the Jacobian matrix J F , which is given by 



and in the inverse way is given by 



where. 



r = J F (x)x 
x = j; 1 (r)r 



(4) 



(5) 
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with e,/ = l,2..,9. 
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Fig. 3. Control system block diagram 
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5. Scalability for the cooperative control of multi-mobile manipulator 

This Subsection proposes a way to generalize the control system associated to the 
coordinated cooperation of three mobile manipulators (virtual structure prism) to a 
coordinated cooperation of n > 3 mobile manipulators. Such proposition is based on the 
decomposition of a virtual 3-dimensional structure of n vertices into simpler components, 
i.e., n-2 prisms. The idea is to take advantage of the control scheme proposed for a virtual 
prism to implement a coordinated cooperative control of n > 3 mobile manipulators using 
the same kinematics transformations presented in previous Section 3, thus not demanding 
to change the Jacobian (Figure 4). 
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Fig. 4. Scalability in the multi-layer control 
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To do that, one should first label the mobile manipulators R t , i = 1,2,3. ..,n and determine 
the leader prism of the whole formation ( R 2 RiR 3 or R 3 R 1 R 2 , paying attention to the 
sequence ABC or ACB). After that, new prisms are formed with the remaining mobile 
manipulators, based on a simple algorithm: a new prism is formed with the last two mobile 
manipulators robots of the last prism already formed and the next mobile manipulator in 
the list of labelled mobile manipulators (in other words, R, +1 RjR: +2 or R; +2 -R -R +1 where 
j = l,2,...,n - 2 represents the current virtual structure prism). Additionally, from previous 
Section 6.3, a set of desired virtual structure variables S F = p F q F fl ¥ z 1F z 2F z 3F 
is assigned to each virtual structure prism. Actually, the number of virtual structure 
variables is the same, but three of the variables has its value defined by the previous 
formation, i.e., 2(n-2) + l, instead of 3(n-2), because it is assumed that 

Sfj = \_Pf h IFj PFj Z 1F h Z 2F,_! z 3F y J ■ 

One point that deserves to be mentioned here is the control signals generated: there will 
always be a redundancy in the virtual structures with more than three mobile manipulators. 
For example, the mobile manipulators R 2 and R 3 , in a virtual structures of four mobile 
manipulators, will receive control signals associated to the errors of the two virtual prisms 
( R 2 R 1 R 3 and R 3 R 2 R 4 , for example). In this work, however, the implementation chosen is one 
in which the mobile manipulators R; +2 will receive control signals only from the controllers 
associated to the j > 2 virtual prisms, while the mobile manipulators R x , R 2 and R 3 will 
receive the signals generated by the controller associated to the leader prism (j = l) . 

Remark 5: the proposed structure is also modular in the horizontal sense, i.e., it grows 
horizontally whenever a new robot is added to the formation. 

Remark 6: the proposed structure is not centralized, since a controller is associated to each 
robot, except for the three first robots, which are governed by a single controller. 

6. Controllers design 

In this section it is presented the design of the controllers for the following control layers: 
Formation Control, Kinematic Control and Adaptive Dynamic Compensation. It is worth 
remark that both the kinematic control and adaptive dynamic compensation are performed 
separately for each mobile manipulator robot. 

6.1 Formation controller 

The Control Layer receives from the upper layer the desired formation pose and shape 
i d = P F S F and its desired variations r rf = P F S F . It generates the pose and shape 

variation references x re f = P F S F , where the subscripts d and ref represent the desired 

and reference signals, respectively. Defining the formation error as r(f) = r d (i)-r(i) and 
taking its first time derivative, the following expression is obtained, 

i = i d-i ( 6 ) 
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Defining r(f) = as the control objective (an equilibrium point of the system), in order to 

prove its stability, it is proposed a controller in the sense of Lyapunov as follows. Defining 
the positive definite candidate function 

V(r) = {r T r >0, 

taking its first time derivative and replacing (6) and i re t = J F x d , assuming -by now- perfect 
velocity tracking, i.e., r = x n t , one gets 

v(i) = i T i = i T (i d -j F x d )- 

Now, the proposed formation control law is defined as 

x d = Jf {i d + *i tanh (K 2 r)) = Jj 1 i ref (7) 

where (^ and K 2 are diagonal positive gain matrix. Introducing (7) into the time derivative 
of V (r) , it is obtained 

V(i) = -i T K 1 tanh(lC 2 i)<0. (8) 

Thus, the equilibrium point is asymptotically stable, i.e. r(t) — > asymptotically. 

Remark 7: Equation (7) represents the desired reference velocity vector for each mobile 
manipulator's end-effector. 

Now, relaxing the assumption of perfect velocity tracking, it is considered a difference S~ (t) 
between the desired and the real formation variations, such as 8- = r re f - r . Then, (8) should 
be written as 

y(r) = r T 8j-r T ^tanh(k: 2 r). (9) 

A sufficient condition for V(i) to be negative definite is, 

|f T K 1 tanh(«: 2 i)| > |i T 8J . (10) 

For large values of r , it can be considered that: ^ 1 tanh(K 2 f) » K x . V(i) will be negative 
definite only if: MA > 8- , thus making the errors f to decrease. Now, for small values of 
f , it can be expressed: fc^ tanh (^ 2 r) ~ H^i , and (10) can be written as, 



r > - 



thus implying that the error r is bounded by, 
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I'll 7\ — r~; ; with 0< $ <1 ( n ) 

Hence, with 8~ (f) * , the formation error r(f) is ultimately bounded by (11). 

6.2 Kinematic controller 

This layer receives the desired position and velocities for each mobile manipulator 
*d=[Ki Ki ■■■ h rf, "■ Kn] and *d = [Ki Ki "■ K i - K »] respectively, 
and it generates the desired kinematic velocities v c =[v cl v c2 ••• v C! •■■ v cn ] for all 
robots. In other words, the desired operational motion of the n mobile manipulators is an 
application (x d (f ) 1 1 e[f ,f^]j . Thus, the problem of control is to find the control vector of 
maneuverability (v c (f)| t e[*o/*f]) to achieve the desired operational motion (7). The 
corresponding evolution of the whole system is given by the actual generalized motion 
(q(f)|fe[f ,f/])- 

The design of the kinematic controller is based on the kinematic model of each mobile 
manipulator robot that belongs to the work team. The kinematic model (1) of the whole 
mobile manipulators can be represented by, 



with 



x(r) = J(q)v(t) 

x(*) = [h 1 (0 h 2 (f) - h,(f) - h„ (*)]%**», 

v(r) = [v a (r) v 2 (f) - v,.(t) - v„(t)]%^»', 
q(0 = [qi(0 ^2(0 - q,(0 - qJOf 6 «"'"', and finally 

J(q) = [Ji(qi) J 2 (q 3 ) •■■ J,(q,) - J„(q„)] T e ^ 3j,x "-"\ 

where n ' represents the dimensions of the generalized spaces associated to the robotic arms 
and to the mobile platforms of all mobile manipulators; i.e., n =n 1 +n 2 --- + n i ... + n n ( see 
Remark 2). 

It is worth noting, that the kinematic controller is performed separately for each robot. 
Hence, to obtain the vector of maneuverability v ; (f) that correspond to the i-ih mobile 

manipulator, the right pseudo-inverse Jacobian matrix J,(q,) is used 

v,=J?n, (12) 
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where, J, = W ! ~'J; (j,W, r J, I , being W ; a definite positive matrix that weighs the control 
actions of the system, 

v,=w < -'j[(j J wr i jj')" 1 h 1 . 

The following control law is proposed for the f-th mobile manipulator. It is based on a 
minimal norm solution, which means that, at any time, the mobile manipulator will attain 
its navigation target with the smallest number of possible movements, 

v cl ■ = J? (V +L K; tanh(4 K, h, )) + (l,. - J?J,.)l Dj tanhfl^D, A,) (13) 

where, h d = h xd h d h zd is the desired velocities vector of the end-effector, h, is the 

control errors vector defined by h, =[h d -h ; ], K,-, D, , L K and L D are positive definite 

diagonal gain matrices that weigh the error vector h, and vector A, . The first term of the 

right hand side in (13) describes the primary task of the end effector which minimizes 

Vj— J,-hJ. The second term defines self motion of the mobile manipulator in which the 

matrix (ij-J/J,-) projects an arbitrary vector A,- onto the null space of the manipulator 

Jacobian ^(J,) such that the secondary control objectives not affect the primary task of the 
end-effector. Therefore, any value given to A, will have effects on the internal structure of 
the manipulator only, and will not affect the final control of the end-effector at all. By using 
this term, different secondary control objectives can be achieved effectively, as described in 
the next subsection. 

In order to include an analytical saturation of velocities in the i-th mobile manipulator, the 
tanh(.) function, which limits the error in h, and the magnitude of the vector A,, is 

proposed. The expressions tanh(L" K .K; h,l and tanhl L" D D, A, I denote a component by 
component operation. 

On the other hand, the behaviour of the control error of the z-th end-effector h, is now 
analysed assuming -by now- perfect velocity tracking v, = v d . By substituting (13) in (12) it 
is obtained 

h, + L K . tanh (I;* . K, h, ) = . (14) 

For the stability analysis the following Lyapunov candidate function is considered 
V ( h, I = i h ; hj > . Its time derivative on the trajectories of the system is 

v(h i .) = -h ] T L Ki tai,h(4K,h ] )<0, 
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which implies that the equilibrium point of the closed-loop (14) is asymptotically stable, 
thus the position error of the i-th end-effector verifies h; (t J — > asymptotically with t -> oo . 

6.2.1 Secondary control objectives 

A mobile manipulator is defined as redundant because it has more degrees of freedom than 
required to achieve the desired end-effector motion. The redundancy of such mobile 
manipulators can be effectively used for the achievement of additional performances such 
as: avoiding obstacles in the workspace and singular configuration, or to optimize various 
performance criteria. In this Chapter two different secondary objectives are considered: the 
avoidance of obstacles by the mobile platform and the singular configuration prevention 
through the system's manipulability control. 

Manipulability 

One of the main requirements for an accurate task execution by the robot is a good 
manipulability, defined as the robot configuration that maximizes its ability to manipulate a 
target object. Therefore, one of the secondary objectives of the control is to maintain 
maximum manipulability of the mobile manipulator during task execution. Manipulability 
is a concept introduced by Yoshikawa (1985) to measure the ability of a fixed manipulator to 
move in certain directions. Bayle and Fourquet (2001) present a similar analysis for the 
manipulability of mobile manipulators and extend the concept of manipulability ellipsoid as 
the set of all end-effector velocities reachable by robot velocities v, satisfying v,- <1 in the 

Euclidean space. A global representative measure of manipulation ability can be obtained by 
considering the volume of this ellipsoid which is proportional to the quantity w called the 

manipulability measure, 



det(j,(q,)j[(q,)) (15) 



Therefore, the mobile manipulator will have maximum manipulability if its internal 
configuration is such that maximizes the manipulability measure w . 

Obstacle Avoidance 

The main idea is to avoid obstacles which maximum height does not interfere with the 
robotic arm. Therefore the arm can follow the desired path while the mobile platform avoids 
the obstacle by resourcing to the null space configuration. The angular velocity and the 
longitudinal velocity of the mobile platform will be affected by a fictitious repulsion force. 
This force depends on the incidence angle on the obstacle a , and the distance d to the 
obstacle. This way, the following control velocities are proposed: 

u obs = Z- 1 {k uohs (d - d)[ft/2 - 1«|]) (16) 

a>*, = r 1 (k wa ,(d -d)8ign{a)[x/2-\a\]) (17) 
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where, d is the radius which determines the distance at which the obstacle starts to be 
avoided, k uobs and k mohs are positive adjustment gains, the sign function allows defining to 
which side the obstacle is to be avoided being sign(0)=l. Z represents the mechanical 
impedance characterizing the robot-environment interaction, which is calculated as 
Z = Is +Bs + K with I, B and K being positive constants representing, respectively, the effect 
of the inertia, the damping and the elasticity. The closer the platform is to the obstacle, the 
bigger the values of a> ohs and u ohs . 

Taking into account the maximum manipulability (15) and the obstacle avoidance (16) and 
(17), the vector A ; is now defined as, 

A i=l- U iobs 0} iobs Kili^ild ~ 0(1 ) Kili^ild-^il) ■- Kina{Oinad ~ Oina)\ C 18 ) 

where k vi [0 id -O^ -being i = l,2,..,n a and k m > - are joint velocities proportional to the 

configuration errors of the mobile robotic arm, in such a way that the manipulator joints will 
be pulled to the desired jd values that maximize manipulability. 

6.3 Adaptive dynamic compensation controller 

The objective of the Adaptive Dynamic Compensation layer is to compensate for the 
dynamics of each mobile manipulator, thus reducing the velocity tracking error. This layer 
receives the desired velocities v c = [v cl v c2 ••• v C! •■■ v cn ] for all robots, and 

generates velocity references v re f = [v re y a v re f 2 ■ ■ ■ v K c i ■■■ v re t n ] to be sent to the 
mobile manipulators. 

The adaptive dynamic compensation is performed separately for each robot. Each one of the 
controllers receive the velocities references v d (f) from the Kinemtic Control layer, and 

generates another velocities commands v refi (f) for the servos of the j-th mobile 

manipulator. 

Thus, if there is no perfect velocity tracking of the i-th mobile manipulator, as assumed in 
Subsection 6.2, the will be a velocity error v ; (f) defined as, v ; =v CI -v ! this velocity error 

motivates to design an adaptive dynamic compensation controller with a robust parameter 
updating law. It is consider the exact model of the i-th mobile manipulator without 
including disturbances (3), 

M,v,+C,v,+G,=v ref , (19) 

Hence, the following control law is proposed for the i-th mobile manipulator is, 

v ref i = *,X, = *,X, + *,X = M,o, + C,v c , + G, + O^. (20) 

where <D(q,v,o) e9^"' x ', \i =[Zi Xi - Xi\ and x, ■= \x\ Xi - Xi] are respectively 
the unknown vector, real parameters vector and estimated parameters vector of the i-th 
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mobile manipulator, whereas X; = x,- - x, represents the vector of parameter errors and 
°i = v c< - + L w tanh(L^K VI v ( ) . 

In order to obtain the closed-loop equation for the inverse dynamics with uncertain model 
(19) is equated to (20), 

M,v, + C,v, + G, = Mfii + C,v d + G, + O,*, 

M,.(a i -v,.) = -C,.v,.-0,.x ( . (21) 

and next, o ; is introduced in (21) 

v, = -M- 1 *,^ -Mfov, -L vi tanh(L-iK v( v ( .) (22) 

A Lyapunov candidate function is proposed as 

V(* i ,Xi) = ^jH i M i v i+ ±xJy i x i (23) 

where y ; e SR x is a positive definite diagonal matrix and H;M,- is a symmetric and positive 

definite matrix. The time derivative of the Lyapunov candidate function on the system's 
trajectories is, 

V(v,., Xi ) = -*J H,M i L vi tanh(L-i K V! . V! ) - vf H^C.-v,. - v^H.O^ + X?Y,X + ^H,M,v, 

Now, recalling that M, (q,) = H^ 1 (M ! +D,) and C i (q i ,v i ) = H?(C i +P i ), 

Due to the well known skew-symmetric property of IM^^C,-), V(v ; ,j(,) reduces to, 

V(v i ,x i ) = -v^H i M ( .L vi tanh(4.K v( v ( )-vfP i v i -vfH,* iXi + x7y,X (24) 

The following parameter-updating law is proposed for the adaptive dynamic compensation 
controller. It is based on a leakage term, or u- modification (Kaufman et al., 1998; Sastry 
and Bodson, 1989). Reference (Nasisi and Carelli, 2003) presented an adaptive visual servo 
controller with a -modification applied to a robot manipulator. By including such term, the 
robust updating law is obtained 

X^Y^Hv -v^r.X,. (25) 

where T, e s Jl x is a diagonal positive gain matrix. Equation (25) is rewritten as 

Xi = Yi X *7HiVi - \?T iXi - Yi X r,Xi (26) 
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Let us consider that the dynamic parameters can vary, i.e., Xi = Zi(f) an d Xi = Xi~X;- 
Substituting (23) in (24), 

^(v ! vxO = -^ H ! M ! L v ! t anh(L- v 1 ,K V! v ! )-vTp ! v ! -xjT,x, -xJ*iXi ~xhiXi ■ (27) 

Considering small values of v, , then L v , tanhf L~ V! K VI v ; ) » K VI v ; . The following constants are 
defined: v r = fc max (r, ■ ) , v y = fc max ( Y , ) , Uv = x{^i), Mmk v p = X, (H,-M,K ¥( . ) + Xi (P;) , where 



%(Z) = , A min \Z Z) is the minimum singular value to Z, fc max (Z) = «Mm ax (Z Z) denotes 
the maximum singular value of Z , and A min (.) and <l max (.) represent the smallest and the 
biggest eigenvalues of a matrix, respectively. Then, V can be rewritten as, 

V{%>Xi) = -Mmk v p pif - Mr \\Xif + v r ||xj|x<|| + y y ||x>||||x>|| . (28) 

Considering £ e SR + in the difference square, 

f 1 II- II ^11 111 1 II- II 2 Til- llll II , /-2II II 2 

— Y- — C Y- = Y- — 2 Y- Y- W + C Y- 

„||A;|| ^||A;|| ^2ll^'ll 11^' llll^' II ' ll^'ll 



can be written as, 

I,- „ 1 ||- ||2 ^ 2 || ,|2 

V. V. < Y- +— — Y- 

||A; ||||Aj || — _ -2 11^' II r, ||A;|| ■ 

By applying a similar reasoning with rj e *R + , it can be obtained 



(29) 



1 2 

ii- mi- n - 1 ii- n2 1 ii- n2 

y- y- < — y- + J — y- nrfi 

||At||||Al|| — _ 2l|A'll ^ ||A;|| ■ W u / 

Substituting (30) and (29) in (28) 

T>f~ ~ \^ II- II 2 II- l|2 f 1 ||- ||2 C 2 II l|2 I | 1 ||- ||2 7 II- l|2 I M\ 

v{Vi'Xi)^-MMK v p\\ v i\\ -Mm +u r\j72\\m + y^ y v r yrfcl + ylMI ( ' 

Equation (31) can be written in compact form as 

^(v,vX,)^^i|v ! f-^|x ! f+^ (32) 

where, s x = {i M k v p > ° / £ 2 = Mr - ^ ~ ^r > ° and 5 = u r ^Xi || + u r ^"llx, || / with £ and V 
conveniently selected. Now, from the Lyapunov candidate function 
V(v i ,x i ) = jvjH l M i v i +jXi\iXi it can be stated that 
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V<AN| 2 +A|Xif (33) 

where p x =\9 M , J3 2 = \9 y , & U = ^x (H.-M,-) , 9 y = lc max ( Y , ) . Then, 

V < -AV + S (34) 

with A = <-^- /-$-[ ■ Since S is bounded, (34) implies that v,(f) and X,(f) are finally 

bounded. Therefore, the o - -modification term makes the adaptation law more robust at the 
expense of increasing the error bound. As 8 is a function of the minimum singular value of 
the gain matrix T t of the a -modification term, and its values are arbitrary, then the error 
bound can be made small. Note that the proposed adaptive dynamic controller does not 
guarantee that X, (f) — > as t — > co . In other words, estimated parameters might converge to 

values that are different from their true values. Actually, it is not required that X, (f) — > in 
order to make v ; (f) converge to a bounded value. 

Remark 8: Note that the updating law (25) needs the H ; matrix. This matrix includes 
parameters of the actuators, which can be easily known and remain constant. Therefore, this 
is not a relevant constraint within the adaptive control design. 

6.4 Stability analysis considering h,(f) and v,(f) 

The behaviour of the tracking error of the end-effector h, (f) of the i-th mobile manipulator 

is now analysed relaxing the assumption of perfect velocity tracking. Therefore, the (14) is 
now written as, 

h, + L K! tanh (4,-K, h, ) = ],v, (35) 

where, v, is the velocity error of the i-th mobile manipulator and J, the Jacobian matrix. It 
is considered a Lyapunov candidate function V\h t \ = -^h, h, and its time derivative, 

y(h,) = h7j,v, -h7L Ki tanh(4,K i h,) . 

A sufficient condition for Vlhj I to be negative definite is 

|h^L K! tanh( I4K, h, )| > (h^v, | . (36) 

Following a similar analysis to the one in Section 6.1, it can be concluded that, if 
r"i Plln thus implying that the error h ; is bounded by, 
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ii- ii llTvll 
h . < " J ' '" - ; with Q<g <1 (37) 

" " ^min( K , ) 

Hence, if v ; ■*■ v CI it is concluded that h ; (f) is ultimately bounded by (37). 

Now finally, generalizing (35) for whole multi-layer control scheme, it is obtained 

x + L K tanh(L^Kx) = Jv (38) 

where x = hj h 2 • ■ ■ h ; • • • h n represents the position error vector of all mobile 
manipulators, K and L K are positive definite diagonal matrices defined as 
K = dwg(K 1 ,K 2 ..,K;..,K n ) and L K = d!ag(L K1 ,L K2 ..,L K! ..,L Kn ), respectively, while the 
function tanh(.) denote a component by component operation. By applying a similar 
reasoning as in (35), it is can concluded that x(f) is bounded by, 

llJvl 
x < — iP-4 — ; with < c < 1 (39) 

For the case of perfect velocity tracking v = v c , i.e., v = 0, it is concluded that x(f)— >0 , 
which implies that r(f)— »0 asymptotically with f— >co, thus accomplishing the control 
objective. Nevertheless, this is not always possible in real contexts, therefore if it is 
considered the velocity error v(f) * , consequently it has that x(f) =£ . This implies that 
the formation error is nonzero 8-(f)^0, i.e., the formation error &\(f) is related with the 
errors x(f) and v(f) . Therefore from (4), (5), (7) and (38) it is obtained the following error 
expression, 

8 f (0 = J F (x)[j(q)v-L K tanh(x)] (40) 

Thus, it is concluded that the adaptive dynamic compensation reduces the velocity error 
v(f) and consequently the error x(f) , hence formation error S-(t) is also reduced. Finally, 
with this results and the conclusion previously obtained from (11), the adaptive dynamic 
compensation controller reduces the height limit of the formation error r(f) . 

7. Simulation result and discussion 

In order to assess and discuss the performance of the proposed coordinated cooperative 
controller, it was developed a simulation platform for multi-mobile manipulators with 
Matlab interface, see the Figure 5. It is important mention that the developed simulator has 
incorporated the dynamic model of the robot. This is an online simulator, which allows 
users to view three-dimensional environment navigation of mobile manipulators. Ours 
simulation platform is based on the MRSiM platform presented by Brandao et al., 2008. 

The f-th 6 DOF mobile manipulator used in the simulation is shown in Figure 5, which is 
composed by a non holonomic mobile platform, a laser rangefinder mounted on it, and a 3 
DOF robotic arm. In order to illustrate the performance of the proposed multi-layer control 
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scheme, several simulations were carried out for coordinated cooperative control of mobile 
manipulators. Most representative results are presented in this section. 




Fig. 5. Mobile manipulator robot used by the simulation platform 

The simulation experiments consist of a team of three or more mobile mamipulator tracking 
a desired trajectory while carrying a payload cooperatively. Also, the obstacle avoidance 
and the singular configuration prevention through the system's manipulability control are 
considered in the simulations. It is assumed the existence of several obstacles, which have a 
maximum height that does not interfere with the robotic arms. That is, the obstacles only 
affect the platform navigation. Hence, the task for each mobile manipulator is divided into 
two subtasks. The first subtask: to carry a payload cooperatively; and the second subtask: 
obstacle avoidance and the singular configuration prevention. 

It is important to remark that for all experiments in this section it was considered that there 
is an error of 30% in dynamic model parameters of each one mobile manipulator robot. 



In the first one it is considered both the position and orientacion of the virtual structure. For 

w =O[rad], 
0.6065[rad] , 



this case, the desired positions of the arm joints are, Robot_l: 
2d = -0.6065[rad] , 3d = 1.2346[rad] . Robot_2 and Robot_3: ld = 0[rad] , 2d 
and 3d = -1.2346 [rad] . Also, the desired virtual structure is selected as, p F =1.75 [m] , 

: 0.3 [m]; while the desired 



q F =1.2[m] and j3 F =1.4 [rad], z lf =0.4 [m] and z 2F 
trajectory for the prism centroid is described by: 



"-IF; 



= 0.2 t + 3.56 



Vf. 



= 3 cos f 0.1 f) + 3 and 



Wv a 



■ + r, 



where 



y = arctan \-jf-/- d y-\ ■ It is worth noting that this trajectory was chosen in order to 

excite the dynamics of the robots by changing their acceleration. The values of the gains 
matrices were adjusted considering the system performance with the dynamic 
compensation deactivated. After this gain setting, the values obtained were used in all 
simulations. 
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Figures 6-9 show the results of the simulation experiment. Figure 6 shows the stroboscopic 
movement on the X-Y-Z space. It can be seen that the proposed controller works correctly, 
where three mobile manipulators work in coordinated and cooperative form while 
transporting a common object. It can be noticed in Figure 6 that there are three triangles of 
different colours representing the upper side of a virtual structure. The yellow triangle 
represents the shape-position of the virtual structure that describes the end-effector of the 
mobile manipulator robots, while the pink triangle represents the location and shape of the 
upper side of the desired virtual structure, and the orange triangle indicates that both 
previously mentioned position-shapes are equal. While, figures 7-9 show that the control 
errors r(f) achieve values close to zero. Figure 7 shows the errors of position and 

orientation of the virtual structure and in Figures 8 and 9 illustrate the errors of the virtual 
structure shape. 
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Fig. 6. Cooperative Coordinated Control of Mobile Manipulators 



In order to show the scalability of the control structure for n > 3 mobile manipulators, the 
following simulations were carried out for coordinated cooperative control of multi-mobile 
manipulators. 

In this context, the second simulation experiment shows a coordinated and cooperative 
control between four mobile manipulators. In this simulation the robots should navigate 
while carrying a payload, following a desired previously defined trajectory. It is considered 
a partially structured environment containing several obstacles in the trajectory. 
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Figures 10 show the stroboscopic movement on the X-Y-Z space, where it can be seen not 
only the good performance of the proposed cooperative control, but also the scalability of 
the proposal for multi-mobile manipulators. The pose and shape errors of the simulation 
experiments are shown in Figures 11 - 14. 



Adaptive Coordinated Cooperative Control of Multi-Mobile Manipulators 



187 



Trajectory of the platform 
Robot l 



^8 



Trajectory of the platform 
Robots 




Fig. 10. 



Cooperative Coordinated Control of Mobile Manipulators 

2.5r 



1.5 



0.5 



-0.5 





_X f _Y f _ V( 


\ 




V 


V 


V 


& \^___ 





10 



20 



30 40 

Time [s] 



50 



60 



Fig. 11 



Position and orientation errors of the triangle(l) 

Pf ^ P f - 




Fig. 12. 



30 40 

Time [s] 

Structure shape errors of the triangleQl^) 



188 



Frontiers in Advanced Control Systems 




30 40 

Time [s] 



Fig. 13. Position and orientation errors of the trianglef2) 



0.15 

0.1 

0.05 



-0.05 

-0.1 

-0.15 

-0.2 

-0.25, 





— Pf Of Pf— z1 f— z2 f z3 f 








— ^ , m .~\ /\ /\ 

















30 
Time [s] 



Fig. 14. Structure shape errors of the triangle(2) 



8. Conclusion 

A multi-layer control scheme for adaptive cooperative coordinated control of n > 3 mobile 
manipulators, for transporting a common object was presented in this Chapter. Each control 
layer works as an independent module, dealing with a specific part of the problem of the 
adaptive coordinated cooperative control. On the other hand, the i-th mobile manipulator 
redundancy is used for the achievement of secondary objectives such as: the singular 
configuration prevention through the control of the system's manipulability and the 
avoidance of obstacles by the mobile platforms. Stability of the system has been analytically 
proved, concluding that the formation errors are ultimately bounded. The results, which 
were obtained by simulation, show a good performance of the proposed multi-layer control 
scheme. 
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1. Introduction 

A repetitive system is one that continuously repeats a finite-duration procedure (operation) 
along the time. This kind of systems can be found in several industrial fields such as 
robot manipulation (Tan, Huang, Lee & Tay, 2003), injection molding (Yao, Gao & Allgower, 
2008), batch processes (Bonvin et al., 2006; Lee & Lee, 1999; 2003) and semiconductor 
processes (Moyne, Castillo, & Hurwitz, 2003). Because of the repetitive characteristic, 
these systems have two count indexes or time scales: one for the time running within the 
interval each operation lasts, and the other for the number of operations or repetitions in 
the continuous sequence. Consequently, it can be said that a control strategy for repetitive 
systems requires accounting for two different objectives: a short-term disturbance rejection 
during a finite-duration single operation in the continuous sequence (this frequently means 
the tracking of a predetermined optimal trajectory) and the long-term disturbance rejection 
from operation to operation (i.e., considering each operation as a single point of a continuous 
process ). Since in essence, the continuous process basically repeats the operations (assuming 
that long-term disturbances are negligible), the key point to develop a control strategy that 
accounts for the second objective is to use the information from previous operations to 
improve the tracking performance of the future sequence. 

Despite the finite-time nature of every individual operation, the within-operation control 
is usually handled by strategies typically used on continuous process systems, such as 
PID ((Adam, 2007)) or more sophisticated alternatives as Model Predictive Control (MPC) 
(Gonzalez et al., 2009a;b). The main difficulty arising in these applications is associated to the 
stability analysis, since the distinctive finite-time characteristic requires an approach different 
from the traditional one; this was clearly established in (Srinivasan & Bonvin, 2007). The 
operations sequence control can be handled by strategies similar to the standard Iterative 
Learning Control (ILC), which uses information from previous operations. However, the ILC 
exhibits the limitation of running open-loop with respect to the current operation, since no 
feedback corrections are made during the time interval the operation lasts. 

In order to handle batch processes (Lee et al., 2000) proposed the Q-ILC, which considers 
a model-based controller in the iterative learning control framework. As usual in the ILC 
literature, only the iteration-to-iteration convergence is analyzed, as the complete input and 



1 In this context, continuous process means one that has not an end time. 
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output profiles of a given operation are considered as fix vectors (open-loop control with 
respect to the current operation). Another example is an MPC with learning properties 
presented in (Tan, Huang, Lee & Tay, 2003), where a predictive controller that iteratively 
improves the disturbance estimation is proposed. Form the point of view of the learning 
procedure, any detected state or output disturbance is taken like parameters that are updated 
iteration to iteration. Then, in (Lee & Lee, 1997; 1999) and (Lee et al., 2000), a real-time feedback 
control is incorporated into the Q-ILC (BMPC). As the authors declare, some cares must be 
taken when combining ILC with MPC. In fact, as read in Lee and Lee 2003, a simple-minded 
combination of ILC updating the nominal input trajectory for MPC before each operation does 
not work. 

The MPC proposed in this Chapter is formulated under a closed-loop paradigm (Rossiter, 
2003). The basic idea of a closed-loop paradigm is to choose a stabilizing control law and 
assume that this law (underlying input sequence) is present throughout the predictions. 
More precisely, the MPC propose here is an Infinite Horizon MPC (IHMPC) that includes 
an underlying control sequence as a (deficient) reference candidate to be improved for the 
tracking control. Then, by solving on line a constrained optimization problem, the input 
sequence is corrected, and so the learning updating is performed. 

1.1 ILC overview 

Iterative Learning Control (ILC) associates three main concepts. The concept Iterative refers 
to a process that executes the same operation over and over again. The concept Learning 
refers to the idea that by repeating the same thing, the system should be able to improve the 
performance. Finally, the concept control emphasizes that the result of the learning procedure 
is used to control the plant. 

The ILC scheme was initially developed as a feedforward action applied directly to the 
open-loop system ( (Arimoto et al., 1984) ; (Kurek & Zaremba, 1993); among others). However, 
if the system is integrator or unstable to open loop, or well, it has wrong initial condition, 
the ILC scheme to open loop can be inappropriate. Thus, the feedback-based ILC has been 
suggested in the literature as a more adequate structure ((Roover, 1996); (Moon et al., 1998); 
(Doh et al., 1999); (Tayebi & Zaremba, 2003)). The basic idea is shown in Fig. 1. 

This scheme, in its discrete version time, operates as follows. Consider a plant which is 
operated iteratively with the same set-point trajectory, y r (k), with k going from to a final 
finite value Tr, over and over again, as a robot or an industrial batch process. During the 
/-th trail an input sequence u'(k), with k going from to a final finite value Tr, is applied to 
the plant, producing the output sequence y'(k). Both sequences, that we will call u' and y', 
respectively are stored in the memory devise. Thus, two vectors with length Tc are available 
for the next iteration. If the system of Fig. 1 operates in open loop, using u' in the (i + l)-th 
trail, it is possible to obtain the same output again and again. But, if at the i + 1 iteration 

information about both, u' and e' = y' — y', where y' = }/ r (0),- ■■ ,y r (Tf) , is considered, 

then new sequences u' +1 and y' +1 , can be obtained. The key point of the input sequence 
modification is to reduce the tracking error as the iterations are progressively increased. The 
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Fig. 1. Feedback-based ILC diagram. Here, continuous lines denote the sequence used 
during the /-th trail and dashed lines denote sequence that will be used in the next iteration. 

purpose of an ILC algorithm is then to find a unique input sequence u 00 which minimizes the 
tracking error. 



The ILC formulation uses an iterative updating formula for the control sequence, given by 



J+l 



f{f,f. 



,y'-',u ! ,u ; - 1 , 



i-l\ 



i,l > 0. 



(1) 



This formula can be categorized according to how the information from previous iteration is 
used. Thus, (Norrlof, 2000) among other authors define, 

Definition 0.1. An ILC updating formula that only uses measurements from previous iteration is 
called first order ILC. On the other hand, when the ILC updating formula uses measurements from 
more than previous iteration, it is called a high order ILC. 

The most common algorithm suggested by several authors ((Arimoto et al., 1984); (Horowitz, 
1993); (Bien & Xu, 1998); (Tayebi & Zaremba, 2003); among others), is that whose structure is 
given by 

V' +1 = Q(2)(V' + C( Z )£ ! ), (2) 

where V = 0, C(z) denotes the controller transfer function and Q(z) is a linear filter. 

Six postulates were originally formulated by different authors ((Chen & Wen, 1999); (Norrlof, 
2000); (Scholten, 2000), among others). 

1. Every iteration ends in a fixed discrete time of duration Tc. 

2. The plant dynamics are invariant throughout the iterations. 

3. The reference or set-point, y r , is given a priori. 

4. For each trail or run the initial states are the same. That means that x* (0) = X° (0), i > 0. 

5. The plant output y(k) is measurable. 

6. There exists a unique input sequence, u°°, that yields the plant output sequence, y, with a 
minimum tracking error with respect to the set-point, e°°. 

Regarding the last postulate, we present now the key concept of perfect control. 



1 94 Frontiers in Advanced Control Systems 

Definition 0.2. The perfect control input trajectory, 

rlT 



U^f 



"rj • ' ■ u Tf-l 



is one that, if injected into the system, produces a null output error trajectory 



e>=\4...4] =[0...0] 



T 



It is interesting to note that the impossibility of achieving discrete perfect control, at least for 
discrete nominal non-delayed linear models, is exclusively related to the input and/or states 
limits, which are always present in real systems and should be consistent with the control 
problem constraints. In this regard, a system with slow dynamic might require high input 
values and input increments to track an abrupt output reference change, producing in this way 
the constraint activation. If we assume a non-delayed linear model without model mismatch, 
the perfect control sequence can be found as the solution of the following (unconstrained) 
open-loop optimization problem 

Tf 2 

u perf _ ar g mm ^ U _ 

u' fc=1 N II 

On the other hand, for the constrained case, the best possible input sequence, i.e., u 00 , is 
obtained from: 



S.M -„2 



ar 



g{min Y^ 4 / s.t. u G U}, 



u' 



k=l 



where U represents the input sequence limits, and will be discussed later. 

A no evident consequence of the theoretical concept of perfect control is that only a controller 
that takes into account the input constraints could be capable of actually approach the perfect 
control, i.e. to approximate the perfect control up to the point where some of the constraints 
become active. A controller which does not account for constraints can maintain the system 
apart from those limits by means of a conservative tuning only. This fact open the possibility 
to apply a constrained Model Predictive Control (MPC) strategy to account for this kind of 
problems. 

1.2 MPC overview 

As was already said, a promising strategy to be used to approach good performances in an 
iterative learning scheme is the constrained model predictive control, or receding horizon 
control. This strategy solves, at each time step, an optimization problem to obtain the control 
action to be applied to the system at the next time. The optimization attempt to minimizes 
the difference between the desired variable trajectories and a forecast of the system variables, 
which is made based on a model, subject to the variable constraints (Camacho & Bordons, 
2009). So, the first stage to design an MPC is to choose a model. Here, the linear model will 
be given by: 

Xk+1 = Ax k + Bu k (3) 
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d k+l = d k (4) 

Vk = Cx k + d k (5) 

where x k £ R" is the state at time k, u k e R m is the manipulated input, y k £ R ( is the 
controlled output, A, B and C are matrices of appropriate dimension, and d k £ R is an 
integrating output disturbance (Gonzalez, Adam & Marchetti, 2008). 

Furthermore, and as a part of the system description, input (and possibly input increment) 
constraints are considered in the following inclusion: 

ueU, (6) 

where U is given by: 

11 = {ue R™ : U m j„ <U< Umax} ■ 

A simplified version of the optimization problem that solves on-line (at each time k) a typical 
stable MPC is as follows: 

Problem PO 

N-l 
min Vv = 



k\k>---' u k+N-l\ki i'=0 



min V k = £ £ (e k+j \ k , u k+j \ k ) + F {e k+N{k 



subjet to: 



e k+j\k = Cx k+j\k + d k+j - yl+j, j = o,...,n, 

x k+j+\\k = Ax k+j\k + Bu k+j\k> ; = 0, . . . , N - 1, 

u k+m eu, 7 = 0,1,. ..,N-1, 

where £(e, u) := ||e||o + ||w|||/ F(e) := ||e||p. Matrices Q and R are such that Q > and 
R > 0. Furthermore, a terminal constraint of the form x k+N i k € Q, where fi is a specific 
set, is usually included to assure stability. In this general context, some conditions should be 
fulfilled by the different "components" of the formulation (i.e., the terminal matrix penalization 
P, the terminal set, fi, etc) to achieve the closed loop stability and the recursive feasibility 
((Rawlings and Mayne, 2009)). In the next sections, this basic formulation will be modified to 
account for learning properties in the context of repetitive systems. 

2. Preliminaries 

2.1 Problem index definition 

As was previously stated, the control strategy proposed in this chapter consists of a basic MPC 
with learning properties. Then, to clarify the notation to be used along the chapter (that comes 
form the ILC and the MPC literature), we start by defining the following index variables: 

• i: is the iteration or run index, where i : = is the first run. It goes from to oo. 



2 Recursive feasibility refers to the guarantee that once a feasible initial condition is provided, the 
controller will guide the system trough a feasible path 
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• k: is the discrete time into a single run. For a given run, it goes from to TV_j (that is, Tt 
time instants). 

• j: is the discrete time for the MPC predictions. For a given run i, and a given time instant 
k, it goes from to H = Tt — k. To clearly state that j represents the time of a prediction 
made at a given time instant k, the notation k + j\k, wich is usual in MPC literature, will be 
used. 

The control objective for an individual run i is to find an input sequence defined by 

iT 



4,-l] (7) 



l '0 
which derives in an output sequence 

y'=[y i o---v , T f }' (8) 



T 



as close as possible to a output reference trajectory 

y r --=[y r T ---y' Tf T } T - W 

Furthermore, assume that for a given run i there exists an input reference sequence (an input 
candidate) given by 

n'" :=[n' T ...u i j f _ 1 T ] T (10) 

and that the output disturbance profile, 



d''= \£ ...d\ 



T 

fl ' 



is known. During the learning process the disturbance profile is assumed to remain 
unchanged for several operations. Furthermore, the value u' T , represents a stationary input 

value, satisfying m' t j = G~ l (y r T — dj )', for every i, with G = [C(I — A)~ l B]. 

2.2 Convergence analysis 

In the context of repetitive systems, we will consider two convergence analyses: 
Definition 0.3 (Intra-run convergence). It concerns the decreasing of a Lyapunov function 
(associated to the output error) along the run time k, that is, V (yjL_i — Vk+l ) — ^ wk+i ~ Vk) 
for k = 1, . . . , Tf_i, for every single run. If the execution of the control algorithm goes beyond Tt, 
with k — >• oo, and the output reference remains constant at the final reference value (y[ = y r T for 
Tt < k < co) then the intra-run convergence concerns the convergence of the output to the final value 

of the output reference trajectory (yjL-i — > yl as k — \ oo). This convergence was proved in (Gonzalez 
et ah, 2009a) and presented in this chapter. 

Definition 0.4 (Inter-run convergence). It concerns the convergence of the output trajectory to 
the complete reference trajectory from one run to the next one, that is, considering the output of a given 
run as a vector ofTr components (y' — > y r as i — > oo). 
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3. Basic formulation 

In this subsection, a first approach to a new MPC design, which includes learning properties, 
is presented. It will be assumed that an appropriate input reference sequence u ! is available 
(otherwise, it is possible to use a null constant value), and the disturbance d\ as well as the 
states x\u are estimated. Given that the operation lasts TV time instants, it is assumed here a 
shrinking output horizon defined as the distance between the current time k and the final time 
TV, that is, H := Tr — k (See Figure 2). Under these assumptions the optimization problem to 
be solved at time k, as part of the single run i, is described as follows: 



Problem PI 



subjet to: 



min V{ = "£ £ (4 +;iJt , u[ +m ) + F (e[ +m ) 



{ u 'k\k'—' U 'k+N s -l\kf j=0 

ei^nu = Cx 
4+/+i|* = A 4+j\k + H +] \k' / = 0, . . . , H - 1 



K k+j\k = ^ X 'k+j\k + d 'k+j - Vk+r ; = o, . . . , H, 



4 +j]k eU, j = Q,l,...,H-l, (11) 

4 + /i* = 4+;+4 + /i*' i = °> 1 ff-i' a 2 ) 

4+j\k = 0, j> Ns, (13) 

where the (also shrinking) control horizon N s is given by N s = min(H, N) and N is the fixed 
control horizon introduced before (it is in fact a controller parameter). Notice that predictions 
with indexes given by k + H\k, which are equivalent to Tr\k, are in fact prediction for a fixed 
future time (in the sense that the horizon does not recedes). Because this formulation contains 
some new concepts, a few remarks are needed to clarify the key points: 



Remark 0.1. In the ith-operation, Tr optimization problems PI must be solved (from k = to k 

pp> 
k+j\k 
i°v 
k\k 



Tr — 1). Each problem gives an optimal input sequence u\ .,,, for j = 0, • • • , H — 1, and following 
the typical MPC policy, only the first input of the sequence, u\,,, is applied to the system. 



Remark 0.2. The decision variables u\ , », are a correction to the input reference sequence u' k+ -(see 
Equation (12)), attempting to improve the closed loop predicted performance. u' k ■ can be seen as the 
control action of an underlying stabilizing controller acting along the whole output horizon, which 
could be corrected, if necessary, by the control actions ui, .... Besides, because of constraints (13), 

wlija is different from zero only in the first N s steps (or predictions) and so, the optimization problem 
PI has N s decision variables (See Figure 2). All along every single run, the input and output references, 
u\ . and y r k ■ , as well as the disturbance d\ . may be interpreted as a set of fixed parameters. 

Remark 0.3. The convergence analysis for the operation sequence assumes that once the disturbance 
appears it remains unchanged for the operations that follow. In this way the cost remains bounded 
despite it represents an infinite summation; this happens because the model used to compute the 
predictions leads to a final input (and state) that matches (yL — dL ). Thus, the model output is 

guided to (y' T — d l T ), and the system output is guided to y r T . 
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k k+N 

Fig. 2. Diagram representing the MPC optimization problem at a given time k. 



3.1 Decreasing properties of the closed-loop cost for a single run 

The concept of stability for a finite-duration process is different from the traditional one since, 
except for some special cases such finite-time escape, boundless of the disturbance effect is 
trivially guaranteed. In (Srinivasan & Bonvin, 2007), the authors define a quantitative concept 
of stability by defining a variability index as the induced norm of the variation around a 
reference (state) trajectory, caused by a variation in the initial condition. Here, we will show 
two controller properties (Theorem 0.1). 1) The optimal IHMPC cost monotonically decreases 
w.r.t time k, and 2) if the control algorithm execution goes beyond Tf with k — ¥ oo , and the 
output reference remains constant at the final reference value (y[ = y r T for k > Tr) then, the 

IHMPC cost goes to zero as k — > oo , which implies that y\ — > y r T as k — > oo. 

Theorem 0.1 (intra-run convergence). Let assume that the disturbance remains constant from one 
run to the next. Then, for the system (3-5), and the constraint (6), by using the control law derived 
from the on-line execution of problem PI in a shrinking horizon manner, the cost is decreasing, that is, 
v f - v k-i + <(eJU'4-i) ^ °'f° r < * < 3> - 1. 
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Furthermore, the last cost of a given operation "i" is given by: 



V T f -l = WTf-l\T f -l' u T f -l\T f -l)+ F ( e Tf\T,-l)> 



and since current and one steps predictions are coincident with the actual values, it follows that: 

V?;U=t(e%U^T f '-i) + HeT f ). (14) 

Proof See the Appendix. □ 

Remark 0.4. The cost VI of Problem PI is not a strict Lyapunov function, because the output horizon 
is not fixed and then, VI (e' k ) changes as k increases (in fact, as k increases the cost becomes less 
demanding because the output horizon is smaller). However, if a virtual infinite output horizon for 
predictions is defined, and stationary values of output and input references are assumed for Tt < oo 
(i.e. u' ss = (C(I — A)~ l B)~ l (y r ss — d l ss ), where d l ss is the output disturbance at Tt), then by selecting 
the terminal cost F{e' ,, ) to be the sum of the stage penalization £(■, •) from Tt to oo, it is possible to 

associate V l k (e[) with a fixed (infinite) output horizon. In this way V k ° p (e l k ) becomes a Lyapunov 
function since it is an implicit function of the actual output error el. To make the terminal cost the 
infinite tail of the output predictions, it must be defined as 

II ■ ■ l|2 II ■ ,- ||2 °° || . ... i|2 

p/J \ _ WCy' 4- it' — 1/ y' -A- r l - TT y ! -A- r l 

r ™W~ r*T f lfc + B ss }fss\\ - \\ x TAk + x ss\\ T - l_i \\ x j Jfc T x ss \\ rTnr 

l=Tf u 

i = 0,l,...,T f -l, 

where x'g S = (I — A)~^Bu l ss and C T PC is the solution of the following Lyapunov equation: 
A T C T PCA = C T PC — C T QC. With this choice of the terminal matrix P, the stability results of 
Theorem 0.1 is stronger since the closed loop becomes Lyapunov stable. 

3.2 Discussion about the stability of the closed-loop cost for a single run 

Theorem 0.1, together with the assumptions of Remark 0.4, shows convergence characteristics 
of the Lyapunov function defined by the IHMPC strategy. These concepts can be extended 
to determine a variability index in order to establish a quantitative concept of stability 
(/3-stability), as it was highlighted by (Srinivasan & Bonvin, 2007). To formulate this extension, 
the MPC stability conditions (rather than convergence conditions) must be defined, following 
the stability results presented in ((Scokaert et al., 1997)). An extension of this remark is shown 
below. 

First, we will recall the following exponential stability results. 

Theorem 0.2 ((Scokaert et al., 1997)). Let assume for simplicity that state reference x r k is provided, 
such that y' k = Cx' k ,for k = 0, . . . , TV — 1, and no disturbance is present. If there exist constants a x , 
&ur b u , c x , c u and d x such that the stage cost £(x, u), the terminal cost P{x), and the model matrices 
A, B and C, in Problem PI, fulfill the following conditions: 

y-WxW 1 < £(x,u) = \\x\\ 2 Q + \\u\\ 2 R < c x .\\x\\ <r + c u .\\u\\ <r (15) 
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H+j\kW ^ b » 11**11' for ) = 0,...,H-1 (16) 

\\Ax + Bu\\ < a x \\x\\ + a u \\u\\ (17) 

F(x) < dxWxW" (18) 
then, the optimal cost VI (x k ) satisfies 

rWxxV <vp(x k ) KrWxtW" (19) 

vf (**)<-?• II **ir (20) 

with 7 = (c x . E^o 1 a f + Nx u- b u + <^x-oc%\, a j = a x-^j-i + a u-K and u Q = a x + a u .b u . 

Proof The proof of this theorem can be seen in (Scokaert et al., 1997). □ 

Condition (15) is easy to determine in terms of the eigenvalues of matrices Q and R. Condition 
(16), which are related to the Lipschitz continuity of the input, holds true under certain 
regularity conditions of the optimization problem. 

Now, we define the following variability index, which is an induced norm, similar to the one 
presented in (Srinivasan & Bonvin, 2007): 

I = max I Lk =° k I 



for a small value of S > 0. With the last definition, the concept of [5 -stability for finite-duration 
systems is as follows. 

Definition 0.5 ((Scokaert et al, 1997)). The closed-loop system obtained with the proposed IHMPC 
controller is intra-run ^-stable around the state trajectory x r k if there exists 5 > such that £ < /3. 

Theorem 0.3 (quantitative f> -stability). Let assume for simplicity that a state reference, x' k , is 
provided, such that y r k = Cx' k , k = 0, . . . , Ti — 1, and no disturbance is present. If there exist constants 
i x , ««/ b u , c x , c u and d x as in Theorem 0.2, then, the closed-loop system obtained with system(3) -(5) 
and the proposed IHMPC controller law is intra-run [5-stable around the state trajectory x r k , with 

7 

Proof See the Appendix. D 

4. IHMPC with learning properties 

In the last section we studied the single-operation control problem, where we have assumed 
that an input reference is available and the output disturbance is known. However, one 
alternative is defining the input reference and disturbance as the input and disturbance 
obtained during the last operation (i.e. the last implemented input and the last estimated 
disturbance, beginning with a constant sequence and a zero value, respectively). In this way, 
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a dual MPC with learning properties accounting for the operations sequence control can be 
derived. The details of this development are presented next. 

4.1 Additional MPC constraints to induce learning properties 

For a given operation i, consider the problem PI with the following additional constraints: 



4 



k+j 



i-r<" 

l k+\\kV } ' 

- 4+v 



k = 0, . . . , T f - 1, j = 0, . . . , H - 1 
k = l,...,T f , j = H 



(21) 



where d{ ■ represents the disturbance estimation. The first constraint requires updating the 
input reference for operation / with the last optimal sequence executed in operation i — 1 (i.e. 
u' = u , for i = 1,2, ■ ■ ■ , with an initial value given by u° := [G~^y' T • • • G~ 1 y' T ]). The 
second one updates the disturbance profile for operation i with the last estimated sequence in 
operation i — 1 (i.e. d ! = d , for i = 1,2, ■ ■ ■ , with an initial value given by d = [0 ■ ■ ■ 0]). 

Besides, notice that the vector of differences between two consecutive control trajectories, 



u' — u' , is given by 5' 



u' ln ■ ■ ■ m' 



, i.e., the elements of this vector are 

U|U lf-l\l f -l^ 

the collection of first control movements of the solutions of each optimization problem PI, for 
fc = 0, ■■■ ,T f -l. 

Remark 0.5. The input reference update, together with the correction presented in Remark 0.2, has 
the following consequence: the learning procedure is not achieved by correcting the implemented input 
action with past information but, by correcting the predicted input sequence with the past input profile 
, which represents here the learning parameter. In this way better output forecast will be made because 
the optimization cost has predetermined input information. Figure 3 shows the difference between these 
two learning procedures. 
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(a) Proposed learning procedures. 



(b) Typical learning procedures. 



Fig. 3. Learning procedures. 

Remark 0.6. The proposed disturbance update implies that the profile estimated by the observer at 
operation i — 1 is not used at operation i — 1, but at operation i. This disturbance update works properly 
when the disturbance remains unmodified for several operations, i.e., when permanent disturbances, or 
model mismatch, are considered. If the disturbance substantially changes from one operation to next 
(that is, the disturbance magnitude or the time instant in which the disturbance enter the system 
change), it is possible to use an additional "current" disturbance correction given by . This correction 
is then added to permanent disturbance profile at each time k of the operation i. 



202 Frontiers in Advanced Control Systems 

4.2 MPC formulation and proposed run cost 

Let us consider the following optimization problem: 

Problem P2 

min VI 

04|it'— '*4+Tf-i|jJ 
subject to (3 - 13) and (21): Run to run convergence means that both, the output error trajectory 
e' and the input difference between two consecutive implemented inputs, S l = u ! u , 
converges to zero as i — ¥ oo. Following an Iterative Learning Control nomenclature, this 
means that the implemented input, u', converges to the perfect control input uP er J . 

To show this convergence, we will define a cost associated to each run, which penalizes the 
output error. As it was said, Tc MPC optimization problems are solved at each run i, that is, 
from k = to k = TV — 1. So, a candidate to describe the run cost is as follows: 

J::=£V> (22) 

J:=0 

where VI represents the optimal cost of the on-line MPC optimization problem at time k, 
corresponding to the run i. 

Notice that, once the optimization problem P2 is solved and an optimal input sequence is 
obtained, this MPC cost is a function of only e\ ,. = I y 1 .,, — y r ,\ = el. Therefore, it makes 
sense using (22) to define a run cost, since it represents a (finite) sum of positive penalizations 
of the current output error, i.e., a positive function of e' . However, since the new run index 
is made of outputs predictions rather than of actual errors, some cares must be taken into 
consideration. Firstly, as occurs with usual indexes, we should demonstrate that null output 
error vectors produce null costs (which is not trivial because of predictions). Then, we should 
demonstrate that the perfect control input corresponds to a null cost. These two properties, 
together with an additional one, are presented in the next subsection. 

4.3 Some properties of the formulation 

One interesting point is to answer what happens if the MPC controller receives as input 
reference trajectory the perfect control sequence presented in the first section. The simplest 
answer is to associate this situation with a null MPC cost. However, since the proposed 
MPC controller does not add the input reference (given by the past control profile) to the 
implemented inputs but to the predicted ones, some care must be taken. Property 0.1, below, 
assures that for this input reference the MPC cost is null. Without loss of generality we 
consider in what follows that no disturbances enter the system. 

Property 0.1. // the MPC cost penalization matrices, Q and R, are definite positive (Q >- and 
R >- 0) and the perfect control input trajectory is a feasible trajectory, then u 1 ' = uP er f <S> Vt f = 
for k = 0, . . . , TV — 1; where 

V l = L l { e k+j\k> u k+j\k) + F { X k+H\k) ■ 
7=0 
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Proof See the Appendix. □ 

This property allow as to formulate the following one: 

Property 0.2. If the MPC cost penalization matrices, Q and R, are definite positive (Q > OandR >- 0) 
and perfect control input trajectory is a feasible trajectory, cost (12), which is an implicit function ofe', 
is such that, e' = <S> /,- = 0. 

Proof See the Appendix. □ 

Finally as trivial corollary of the last two properties, it follows that: 

Property 0.3. If the MPC cost penalization matrices, Q and R, are definite positive, then u' = 
uP erf ^ ji = o otherwise, u 1 ' £ uP er f =>• f £ 0. 

Proof It follows from Property 0.1 and Property 0.2. □ 

4.4 Main convergence result 

Now, we are ready to establish the run to run convergence with the following theorem. 

Theorem 0.4. For the system (3)-(5), by using the control law derived from the on-line execution of 
problem P2 in a shrinking horizon manner, together with the learning updating (21), and assuming 
that a feasible perfect control input trajectory there exists, the output error trajectory e l converges to 
zero as i — \ oo. In addition, S' converges to zero as i — I °o which means that the reference trajectory 
u' converges to uP er i . 

Remark 0.7. In most real systems a perfect control input trajectory is not possible to reach (which 
represents a system limitation rather than a controller limitation). In this case, the costs Vj ( will 
converge to a non-null finite value as i — > oo ,and then, since the operation cost f is decreasing (see 
previous proof), it will converge to the smallest possible value. Given that, as was already said, the 
impossibility to reach perfect control is exclusively related to the input and/or states limits (which 
should be consistent with the control problem constraints), the proposed strategy will find the best 
approximation to the perfect control, which constitutes an important advantage of the method. 

Remark 0.8. In the same way that the intra-run convergence can be extended to determine a 
variability index in order to establish a quantitative concept of stability (^-stability), for finite- 
duration systems (Theroem 0.3); the inter-run convergence can be extended to establish stability 
conditions similar to the ones presented in (Srinivasan & Bonvin, 2007). 

5. Ilustrative examples 

Example 1. In order to evaluate the proposed controller performance, we consider first a linear 
system (Lee & Lee, 1997) given by G(s) = l/15s 2 + 8s + 1. The MPC parameters were tuned 
as Q = 1500 , R = 0.5 and T = 1. Figure 4 shows the obtained performance in the controlled 
variable where the difference with the reference is undistinguished. Given that the problem 
assumes that no information about the input reference is available, the input sequence u and 
u are equals. 
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Time [k) 

Fig. 4. Reference, output response according to the input variables u and u 

1 




Time (k) 
Fig. 5. Normalized MPC cost function. Here, the normalized cost function is obtained as 
V k/ V kmax- 

The MPC cost function is showed in Fig. 5. According to the proof of Theorem 0.1 (nominal 
case), this cost function is monotonically decreasing. 

Example 2. Consider now a nonlinear-batch reactor where an exothermic and irreversible 
chemical reaction takes place, (Lee & Lee, 1997). The idea is to control the reactor temperature 
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by manipulating the inlet coolant temperature. Furthermore, the manipulated variable has 
minimum and maximum constrains given by: Tc m [ n < Tc < Tc max , where Tc m [ n = — 25 [°C], 
Tc max = 25 [°C] and, Tc is written in deviation variable. In addition, to show how the 
MPC controller works, it is assumed that a previous information about the cooling jacked 
temperature (u = Tc) is available. 

Here the proposed MPC was implemented and the MPC parameters were tuned as, Q = 1000 
, R = 5 and T = l[min]. The nominal linear model used for predictions is the same proposed 
by (Adam, 2007). 

Figure 6 shows both the reference and the temperature of the batch reactor are expressed in 
deviation variable. Furthermore, the manipulated variable and the correction made by the 
MPC, u are shown. 

Notice that, 1) the cooling jacked temperature reaches the maximum value and as a 
consequence the input constraints becomes active in the time interval from 41 minutes to 
46 minutes; 2) similarly, when the cooling jacked temperature reaches the minimum value, 
the other constraint becomes active in the time interval from 72 minutes to 73 minutes; 3) 
the performance is quite satisfactory in spite of the problem is considerably nonlinear and, 4) 
given that it is assumed that a previous information about the cooling jacked temperature is 
available, the correction u is somewhat smaller than u (Fig. 6). 




i 1 1 1 r 

Batch Reactor Temperature ~ — _ 



Reference Temperature 

J I I I L 





40 SO 

Time ([min]) 

Fig. 6. Temperature reference and controlled temperature of the batch reactor. Also, the 
cooling jacked temperature (u) and the correction (u) are showed. 

Example 3. In order to evaluate the proposed controller performance we assume a true and 
nominal process given by (Lee et al, 2000; Lee & Lee, 1997) G(s) = l/15s 2 + 8s + 1 and 
G(s) = 0.8/12s 2 + 7s + 1, respectively. The sampling time adopted to develop the discrete 
state space model is T = 1 and the final batch time is given by Ti = 90 T. The proposed 
strategy achieves a good control performance in the first two or three iterations, with a rather 
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Fig. 7. Output and input responses. 



reduced control horizon. The controller parameters are as follows: Q = 1500, R = 0.05, N = 5. 
Figure 7 shows the output response together with the output reference, and the inputs u' and 
u l , for the first and third iteration. At the first iteration, since the input reference is a constant 
value (u' T _, = 0), u' and u' are the same, and the output performance is quite poor (mainly 
because of the model mismatch). At the third iteration, however, given that a disturbance 
state is estimated from the previous run, the output response and the output reference are 
undistinguishable. As expected, the batch error is reduced drastically from run 1 to run 3, 
while the MPC cost is decreasing (as was established in Theorem 0.1) for each run (Fig. 8a). 
Notice that the MPC cost is normalized taking into account the maximal value ( VI / V^ ax ) , 




(a) Error and MPC cost. (b) Norm of the iteration error. 

Fig. 8. Error and MPC cost, and Norm of the iteration error for the example 3. 
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where V^ ax « 1.10 6 and V^ ax ~ 286.5. This shows that the MPC cost } l decrease from one 
run to the next, as was stated in Theorem 0.4. Finally, Fig. 8b shows the normalized norm of 
the error corresponding to each run. 

6. Conclusion 

In this paper a different formulation of a stable IHMPC with learning properties applied to 
batch processes is presented. For the case in which the process parameters remain unmodified 
for several batch runs, the formulation allows a repetitive learning algorithm, which updates 
the control variable sequence to achieve nominal perfect control performance. Two extension 
of the present work can be considered. The easier one is the extension to linear-time- variant 
(LTV) models, which would allow representing the non-linear behavior of the batch processes 
better. A second extension is to consider the robust case (e.g. by incorporating multi model 
uncertainty into the MPC formulation). These two issues will be studied in future works. 

7. Appendix 
Proof of Theorem 0.1 

Proof Let u^ := {^\\k-V ■ ■ '^ s -2\k-V Q ojandx^ := {<!' 1|fc _ 1 , .. •,<+ T/ ]*-l } 

be the optimal input and state sequence that are the solution to problem PI at time k — 1, with 
k = 1, • • • , Tc — N (that means that the last N optimization problem of a given run i are not 
considered). The cost corresponding to these variables are 



yf - r Ns_1 1 (e i ° v ' if' \ + T H - 1 ( (e fl " 6) + F ( x'" P ' 



Lj=0 * y ! l c +j-l\k-l' u k+j-l\k-l) +r \ x T f \k-l) 



(23) 



Notice that at time k — 1, H = Tc — k + 1, since H is a shrinking horizon. Now, let Uj. : = 

1 "jtlJt-1 ' ' ' ' ' U k+N -2lSr-l ' 0' • • • ' f be a feasible solution to problem PI at time k. Since no 

new input is injected to the system from time k — 1 to time k, and no unknown disturbance is 

considered, the predicted state at time k, using the feasible input sequence, will be given by 

^' /ras ._ l v i°l" r i°>" Y i"f \ _ I r i°i" J '" J '" X Thprt fop 

x k ■- (//cjfc-l'- ■•' x Jc+H-ljJc-l'' l Jc+H|Jc-lJ ~ \*k\k-V"'*k+Tf\k-V*k+T f +l\k-iy lnen ' lIle 

cost at time k corresponding to the feasible solution u' is as follows: 



v k - Lj =0 * ^ e jt +; ] jc-i ' u k+j\k- 1) + Lj =Ns « \e k+ j\ k _ v u) "I" r ^e fc+J j| fc _i 



pH-1 p (j'Pt jrfP> ) , r (j°P> ) 

Lj=0 l \ e k+i\k-V U k+j\k-\) + t \ e T f \k-\) 



(24) 



Notice that now H = Tc — k, because predictions are referred to time k. Now, subtracting (23) 

from (24) we have 

if**, ,.,.- _ fm i°p> \ 

V k V k-l~ « \ e k-l\k-l' U k-l\k-l) • 
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This means that the optimal cost at time k, which is not grater than the feasible one at the same 
time, satisfies 

jOpt M „ / /Off _jl.pt 



Finally, notice that ei_, ,, _, and u',_ , ,, , represent actual (not only predicted) variables. Thus, 
we can write 

Vr-v£ 1+ l(C v lCi)<0. (25) 

This shows that, whatever the output error is different from zero, the cost decreases when 
time k increases. 

Finally, the decreasing property for k = Tr — N + 1, • • • , Tr — 1, and the last part of the 
theorem, can be proved following similar steps as before (i.e., finding a feasible solution). 

□ 



Proof of Theorem 0.3 

Proof From the recursive use of (25), together with (15), (19) and (20), we have 

v i°t" < v i"i" _ i ( Y i jji\ <7 - ii v i iicr _ m-( ii<r _ r^- _ ^Wyi \\<r 
v k+l- v k ' \ x k' u k) ^ 7-11**11 7-11**11 -17 7) \\ x kW ' 

for k = 0, . . . , Tt — 2. So we can write: 



=i lie 



Tf-1 


r t,-\ 




E < < 

k=0 


r+ £ (7- 

H = l 


-7)" 



Therefore, 



zlr 1 vr'b + &i(7- 1 r] 



V*° pt - 7 

since 7. || Xq \\ a is a lower bound of Vq P (that is, 7. || x$ \\ a < Vq P ). 
Finally, 

7 

n 

Proof of Property 0.1 

Proof <=) Let us assume that VI = , for k = 0,...,Tc — 1. Then, the optimal predicted 
output error and input will be given by el .,, = 0, / = 0,...,Tc and u\ .,, = 0, for j = 

0, ..., Tt — 1, respectively. If e\ .,, adn ul ... = simultaneously, it follows that ui = m? for 
fc = 0, . . . , Tr — 1, since it is the only input sequence that produces null predicted output error 
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(otherwise, the optimization will necessarily find an equilibrium such that eJL, ,iJ > and 



*k\k 



> 0, provided that Q >- and R >- by hypothesis). Consequently, u' = vJ' er ' . 



=S> ) Let us assume that uj. = uf . Because of the definition of the perfect control input, the 
optimization problem without any input correction will produce a sequence of null output 
error predictions given by 

4\ k = ° 

4+i|/c = Cx 'k+i\k ~ yl+i = c [ Ax 'k\k + Bu k \ ~ yl+i = ° 



4+r,|* = Cx k + T f \k - yl + T f = C [A T fx' klk + ABu["f ■ ■ ■ Buf^] - y' k+Tf = 0. 

Consequently, the optimal sequence of decision variables (predicted inputs) will be u. ... = 
for k = 0, . . . , Tf — 1 and j = 0, . . . , TV — 1, since no correction is needed to achieve null 
predicted output error. This means that VI =0 for k = 0, . . . , TV — 1. □ 

Proof of Property 0.2 

Proof =>) Let us assume that e' = 0. This means that el ,, = 0, for k = 0, . . . , Tr. Now, assume 

that the input reference vector is different from the perfect control input, u" ^ \xP er f , and 
consider the output error predictions necessary to compute the MPC cost V' k : 

A\k = ° 
e i+\\k = Cx k+i\k - yl+i = c [ Ax ' k \k + Bu k + B K\k] - yl+i 



Since u l is not an element of the perfect control input, then Axl,. + Bu l k ^ 0. Consequently, 
(assuming that CB is invertible) the input m!,, necessary to make el . ,, = 0, will be given by: 

4\k = (cBr 1 (y r k + i-c[Ax i klk + B4}[ 



which is a non null value. However, the optimization will necessary find an equilibrium 
solution such that ||ei , -.,, || > and \\uL k < ttli JL since Q > and R > by hypothesis. 

This implies that 3 e l k+l i k = e Jt + nt; + i 7^ 0' contradicting the initial assumption of null output 
error. 

From this reasoning for subsequent output errors, it follows that the only possible input 
reference to achieve e' = will be the perfect control input (u ! = vJ' er f). If this is the case, it 
follows that Vf* = , for k = 0, ..., Tf (Property 0.1), and so, /' = 



3 Note that for the nominal case is el , , , = e\ ,,,, 

k+l\k-\-l k-\-l\k 
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Let us assume that /' = 0. Then, VI = 0, which implies that e'. 



0, for k = 0, ..., T 



f 



and for j = 0, ..., Tc . Particularly, e\ , = 0, for k = 0, ..., Tc, which implies e' = 0. □ 



Proof of Theorem 0.4 

Proof The idea here is to show that VI < VjT for k = 0, . . . , Tc — 1 and so, /, < /,_i . First, 
let us consider the case in which the sequence of Tc optimization problems P2 do nothing at a 
given run i. That is, we will consider the case in which 



'0|0 



*T/-i|iy-i 



[0...0] T , 



for a given run i. So, for the nominal case, the total actual input will be given by 



u< = u'- 1 = [u' - lJ 






TlT 



' ( 0|0 



jopt' 



1 T 



and the run cost corresponding to this (fictitious) input sequence will be given by 



Ji 



/c=l 



where 



n-- 



L 



. 1°'" —i 
e k+j\k+j' U k+j\k+j 



V 



) 



H-l 



«;;v,.»)+ f W 



rH\k+H-l) 



H-l 

E' 

,/=o 



£3) +*(*&*) 



(26) 



Since the input reference, u\ ■, that uses each optimization problems is given by id , ■ 



_jop[ 



., then the resulting output error will be given by e 



opt 

k+j\k+j 



e k 



j. for ; = 0, ..., H. 



In other words, the open loop output error predictions made by the MPC optimization at 
each time k, for a given run i, will be the actual (implemented) output error of the past run 
i — 1. Here it must be noticed that &7r- refers to the actual error of the system, that is, the 

error produced by the implemented input "l+i'-i = M t+ ■_i|tx-_r Moreover, because of the 
proposed inter run convergence constraint, the implemented input will be ut~_i, for j > H. 

Let now consider the optimal MPC costs corresponding to k = 0, ..., Tf — 1, of a given run 
i — 1. From the recursive use of (12) we have 



K 



_l°p f 






i opt 

< V'~ l 
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^r+^WO^ 1 " 



Then, adding the second term of the left hand side of each inequality to both sides of the next 
one, and rearranging the terms, we can write 

\-T + *> (4;- 2 < «r7- 2 ) + •••+< (4 _1 < "(r 1 ) < n~ l ° e ' ( 27 ) 

■ -iopt 

From (14), the cost VL _, , which is the cost at the end of the run i — 1, will be given by, 

+ f(x' t T 1 ). (28) 



V'- 1 :' =l(eir-\,u i - 1 



Tf-l ~ *• \fT f -V*Tf-\ 
Therefore, by substituting (28) in (27), we have 

F (4; 1 ) + e (j^-v^-i) +■■■ + ' (4 _1 <"o _1 ) < vf" 1 * ( 29 ) 

Now, the pseudo cost (26) at time fc = 0, VL can be written as 

= E 1 ' (t 1 '^-" 1 ) +F K; 1 ) - e' IN" 1 ! (30) 

7=0 ;=o 

and from the comparison of the left hand side of inequality (29) with (30), it follows that 

7 >- 1 „ 

v o - v o Aj "; 

7=0 
Repeating now this reasoning for fc = 1, ..., TV — 1 we conclude that 

n=vt r,,t - e IH -1 !' fc=o T /- : 

/=* 

Therefore, from the definition of the run cost J i we have 4 

T f -lTf-l 

Ji<h-i- E E IN -1 II- (3i) 

)t=0 j=k 



4 Notice that, if the run i implements the manipulated variable u'- = u, + «j l , j = 0, 1, . . . , TV — 1 and 
m'. 7^ for some /'; then, according to 31 /, < /(_i. Unnaturally, to have found a non null optimal 
solution in the run i — 1 is sufficient to have a strictly smaller cost for the run i. 
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The MPC costs VjJ is such that Vf r < VI, since the solution u', .,. = 0, for j = 0, . . . , H is a 
feasible solution for problem P2 at each time k. This implies that 



li < /'• 02) 



From (31) and (32) we have 



Tf-lTf-l 
},<Ji<h-i- E E Ik" 1 !. (33) 

k=0 j=k 

which means that the run costs are strictly decreasing if at least one of the optimization 
problems corresponding to the run i — 1 find a solution «!""_ .,, 7^ 0. As a result, two options 
arise: 

I) Let us assume that u' 7^ xxP er f . Then, by property 0.3, /' 7^ and following the reasoning 
used in the proof of Property 0.2, u'- 7^ 0, for some 1 < j < Tc. Then, according to 33, 

Ji+1 < Ji+i < J, - EEo 1 E^ 1 |«j~ 1 1) with |uj| > for some 1 < / < T f - 1. 

'r . 1||-|| T- lll-l 

The sequence /' will stop decreasing only if E;=n "/' = 0- ^ addiction, if YLjLn "; = 0/ 
then u' = vJ' er J, which implies that /, = 0. Therefore: lim,^.,^ /, = 0, which, by Property 0.2 
implies that lim,-^.,^ e, = 0. 

Notice that the last limit implies that lim/^oo S l = and consequently, lim^oo u' = vtP er 5 . 

II) Let us assume that u' = xtP er f . Then, by Corollary 0.3, /,- = 0, and according to (33), 
/, + l = 7, + i = /, = 0. Consequently, by Property 0.2, e' = 0. □ 
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1. Introduction 

Actually the development of control systems in embedded systems presents a great advantage 
in terms of easy design, immunity to analog variations, possibility and implement complex 
control laws and design a short time (Mingyao Ma et al., 2010). One of the devices that 
allows embedded systems arrangements are field-programmable gate array (FPGA). Several 
advantages of using FPGAs in industrial applications can be seen in (Joost & Salomon, 2005). 

The use of FPGAs to implement control laws of various systems can be observed in different 
articles. In (Hwu, 2010) performance a technique based on a field programmable gate array 
to design PID controller applied to the forward converter to reduce the effect of input voltage 
variations on the transient load response of the output converter. The main characteristic of 
this technique is the on-line tuned parameters of the PID controller. To validate the topology 
implemented, they designed a forward converter with an input voltage of 12V, and output 
dc voltage of 5.12V with a rated output DC current of 10A and a switching frequency at rated 
load of 195 kHz. The results show than the measured transient load response has no oscillation 
with on-line tuning applied to the controller. 

In the work of LI et al. (Bo Li et al., 2011) presents a digital pulse- width-modulator based 
sliding-mode controller and FPGA for boost converter. The proposed model they used 
was higher order delta-sigma modulator. The problem with this modulator is the stability 
problem. To resolve this problem they implemented a Multi-stage-noise shaping delta-sigma 
DPWM (MASH sigma-delta DPWM). To verify the function of the proposed controller they 
implemented a boost converter connected to a Virtex-II Pro XC2VP30 FPGA with and Analog 
to digital converter as interface. The experimental results show than the MASH sigma-delta 
DPWM has a faster recovery time in load changes, compared with a PID controller. 

In (Mingyao Ma et al., 2010) proposed a FPGA-based mixed-signal voltage-mode controller 
for switching mode converters. The architecture of the scheme consists of a DPWM generation 
with a PID controller implemented on FPGA, a DAC and a comparator. The switching mode 
converters state variables are digitalized via an ADC to the PID controller. The control signal 
goes to the DPWM module to generate the PWM waveforms. They implemented the PID and 
the DPWM on a Cyclone II series EP2C25, in other hand; they implemented a single phase 
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full-bridge inverter like the switching mode converter to test the architecture of the controller. 
Their architecture allows integration of a control system in FPGA. 

An implementation of PID controller on FPGA for low voltage synchronous buck converter 
is presented in (Chander et al., 2010). They use MATlab/Simulink for the PID controller 
design to generate the coefficients of the controller. They did a comparison between 
different coefficients to obtain a reasonable controller for the converter. The architecture was 
implemented in FPGA Virtex-5 XC5VLX50T 

In this article, we will focus on the PID average output feedback controller, implemented in an 
FPGA, to stabilize the output voltage of a "buck" power converter around a desired constant 
output reference voltage. The average control inputs are used as a duty ratio generator in 
a PWM control actuator. The architecture control, used for the classical PID control, has the 
following features: 

• The PWM actuator is implemented through a triangular carrier signal and a comparator. 
The main function of this modulator is the average signal conversion to a pulsing signal 
that activates and deactivates the converter power transistor, at a switching frequency of 
48kHz. 

• The processing time control for the PID is 20.54^s. This processing time were achieved 
thanks to the parallel execution of units modeled within a FPGA Monmasson & Cirstea 
(2007)-Rogriguez-Andina et al. (2007). 

• The output voltage is obtained through an Analog to Digital Converter (ADC), which is 
the only additional hardware needed to operate to the controllers. The used ADC is the 
ADC0820, which is an 8 bits converter. 

The rest of the document is organized as follows: section 2 presents the mathematical model 
of the "buck" converter. The design of the PID control is shown in the section 3, while 
the simulation of the PID control design is presented in section 4. The architecture of the 
implemented control is found in section 5. The experimental results of the implementation 
of the FPGA based controller, are found in section 6. Finally, the conclusions of this work are 
given section 7. 

2. The "buck" converter model 

Consider the "buck" converter circuit, shown in Fig. 1. The system is described by the 
following set of differential equations: 

L— = -v + Eu 

y = v 

where ii represents the inductor current and Vq is the output capacitor voltage. The control 
input u, representing the switch position function, takes values in the discrete set 0, 1. The 
system parameters are constituted by: L and C which are, respectively, the input circuit 
inductance and the capacitance of the output filter, while R is the load resistance. The external 
voltage source exhibits the constant value E. The average state model of the "buck" converter 
circuit, extensively used in the literature (a) Linares & Sira, 2004; b) Linares & Sira, 2004; 
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Linares et al., 2011; Sira & Agrawal, 2004) may be directly obtained from the original switched 
model, (1), by simply identifying the switch position function, u, with the average control, 
denoted by u av . Such an average control input is frequently identified with the duty ratio 
function in a Pulse Width Modulation implementation. The control input u av is restricted to 
take values in the closed interval [0, 1]. From (1), the "buck" converter system is clearly a 
second order linear system of the typical form: x = Ax + bu and y = ex. 
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Fig. 1. The electrical circuit of the "buck" converter. 
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(2) 



J = [0 1] 
Hence, the Kalman controllability matrix of the system C = [b, Ab\, is given by: 



C 



LC 



(3) 



The determinant of the controllability matrix is ( -k^ ^ 0). Therefore, the system is controllable 
(Dorf & Bishop, 2011), now we design a classic PID control in the following section. 

3. PID controller design 

The FPGA implementation of a classical Proportional Integral Derivative (PID) controller 
was designed based on the corresponding transfer function of the converter (Ogata, 2010), 
obtained from the average model given in (1), is 



Vo(s) 

Uav(s) 



LC 



s 2 + -J-s- 



LC 



(4) 



While the transfer function of the PID controller, is: 



Fpi D (s)=K p (1+ — + T d s) 

The block diagram of the PID controlled system is shown in Fig. 2. 
The closed loop transfer function is readily found to be 



(5) 



H(s) 



(K p T d T iS 2 + K p T iS + K p )(^) 

( 1 I EK,Ti\„2 | (1+EM EK P 

^ RC "l" LC '" "^ LC "^ LCT; 



(6) 
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Fig. 2. PID control in closed loop. 

The closed loop characteristic polynomial of the PID controlled system is then given by 

EK„ 



3 , / * , EK P T d^ 2 , (1 + E-Kp) . 

s + ( — )s H — s H — 

v RC LC ; LC LCT; 







(7) 



The coefficients K„, T, and T^ are chosen so that (7) becomes a third order Hurwitz polynomial 
of the form (Dorf & Bishop, 2011; Ogata, 2010): 



p(s) = (s 2 + 2f co„s + w n 2 )(s + a) 



(8) 



Equating the characteristic polynomial coefficients (7) with those of the desired Hurwitz 
polynomial (8), we obtain the following values of the parameters for the PID controller, 



K p 
Ti 



2£w„aLC + to n 2 LC — 1 

EKp 

LCoaVn 2 
LC , , 1 , 

— ( a + 2^„--) 



(9) 



4. PID controller cosimulation 

In this section, we develop the simulation of the PID controller. This simulation is performed 
using Matlab/Simulink, ModelSim and PSim Software. 

The cosimulation in Matlab/Simulink creates an interface between Matlab and Matlab 
external program, i.e., the cosimulation allows the interaction of an external simulator with 
Matlab tools. The cosimulation provides a fast bidirectional link between the hardware 
description language (HDL) simulator, and Matlab/Simulink for direct hardware design 
verification Matlab (2008). 

Figure 3 shows the scenario between ModelSim and Matlab to obtain the cosimulation. The 
block that allows interaction with the HDL simulator is called "EDA Simulator Link MQ". 

The PSIM software includes an application called SimCoupler that presents an interface 
between PSIM and Matlab Simulink for cosimulation. With the module SimCoupler part of the 
system can be implemented and simulated in PSIM, and the rest of the system in Simulink. 
With this tool we can access to the broad features of Psim simulation, and the capabilities of 
Simulink simulation in a complementary way Psim (2006). 

The module SimCoupler consist of two parts: the link nodes in PSim, and the SimCoupler 
model block in Simulink. In this work we use the module SimCoupler to simulate the buck 
converter in Psim, while in matlab and through another cosimulation part of the PID control. 
Figure 4 shows the buck converter circuit in Psim, in this figure one can observe that the circuit 
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Fig. 3. Cosimulation between Matlab and ModelSim. 

input is the signal coming from the PWM control in Simulink, this input signal is connected 
using the In link node. Because the system is feedback, and the feedback signal is the output 
voltage of the buck converter (V ), we use the Out link node to send the output voltage to 
Simulink. 
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Fig. 4. Buck converter circuit for cosimulation with Simulink. 

In Simulink we choose the S-function SimCoupler library and the SimCoupler block are added 
to the design. After adding the block, in its properties is chosen the file path for cosimulation 
in Psim, a window will automatically appear as shown in Fig. 5 showing all inputs and 
outputs of the circuit in PSIM, in this case, according to the diagram there are one input and 
one output on the circuit, the output voltage of the buck converter V . While that the input is 
the PWM signal. 

Once set the block are automatically displayed input and output signals in the block for 
subsequent cosimulation, as shown in Fig. 6. 

Before simulating the final system, we proceed to simulate the performance of open-loop buck 
converter, for this, we just simulate the circuit in Psim. Figure 7 shows the output voltage in 
simulation for open-loop. The response presents a overshoot of 100% however are able to 
stabilize around 45ms. 

On the other hand, we simulate the PID control with the tools of differentiation and 
integration of Simulink. Figure 8 shows the performance of the PID controller with 
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Fig. 5. Inputs and Outputs of the PSIM circuit. 
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Fig. 6. Simulink block with the inputs and outputs of the PSIM circuit. 
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Fig. 7. Output voltage of buck converter in open-loop. 

cosimulation between Matlab/ Simulink and Psim. The PID control stabilizes the output 
voltage signal in a time of approximately 18ms, greatly decreasing the overshoot presented 
at the open-loop response. 

Figure 9 shows a cosimulation for the final system with a desired output voltage of 4V, and 
shows that in the transient response has not overshoot, however, the settling time is about 23 
ms, what is intended to improve with the experimental results. Also the Fig. 10 shows the 
output voltage for a desired voltage of 18 V, which shows that it has a maximum overshoot 
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Fig. 8. Output voltage of buck converter with PID controller in cosimulation. 

of 7.6 %, and a maximum error of 0.15 V. According to these simulations, we proceed to 
implement the system on a FPGA NEXYS2 board. 




0.005 



0.010 



0.020 



0.025 



0.015 

Time [s] 

Fig. 9. Output voltage of buck converted with a desired voltage of 4 V in cosimulation. 
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5. Discrete PID controller implemented on the FPGA 

In this section, we explain the hardware implementation of the discrete PID controller. 
For this purpose, we used the Xilinx ISE Design Suite 12.2 EDA (electronic design 
automation) -software tool and the Spartan 3E board EDA-hardware tool, it includes a Xilinx 
Spartan-3E1600 FPGA. 
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Fig. 10. Output voltage of buck converted with a desired voltage of 4 V in cosimulation. 

Now, we must define an efficient design methodology and the abstraction level to model the 
system, and choose an appropriate sampling period and the suitable format for coefficients 
and variables. 

The PID controller design is based on a hierarchical and modular approach using Top-Down 
methodology (Palnitkar, 2003), where the modules can be defined with diverse levels of 
abstraction. Thus, for this design the schematic description was chosen as top level and 
the controller components were modeled with the VHDL hardware description language 
(using a behavior level modeling). Previous analysis and simulations showed that due 
to the range of results generated by the operations involved in the discrete controller is 
necessary to use a floating point format; for this intention, the IEEE Standard for Binary 
Floating-Point Arithmetic, IEEE Std 754-1985 (IEEE, 1985) was chosen. Now, based on 
top-down methodology, an initial modular partitioning step is applied on the FPGA-based 
PID controller, this process generate four components, Clock manager, ADC control, Control 
law and PWM generator (see Fig. 11). 

The PID controller work with a frequency of 50 MHz (Clk_PID). The Clk_main signal is 
generated from Clk_main signal by the Clock manager component. The principal element of 
this component is the Digital Clock Manager (DCM). The DCM is embedded on the Spartan3E 
FPGA's families and it provides flexible complete control over clock frequency, maintaining 
its characteristics with a high degree of precision despite normal variations in operating 
temperature and voltage. The DCM provides a correction clock feature, ensuring a clean 
Clk_PID output clock with a 50% duty cycle. 
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Fig. 11. Block diagram of FPGA-based PID controller. 

In order to increase the performance of the control system, we proposed pipeline architecture 
for the PID controller. Therefore, enable signals of pipeline registers (Stage_enable0..9) are 
required. These signals are also generated by the Clock manager component. 

In addition, the clock manager component generates the frequency required by the PWM 
for its operation (Clk_PWM). The Clk_PWM signal is derived from the Clk_main by a Digital 
Frequency Synthesizer (DFS) included in the DCM. The frequency of the Clk_PWM is 25 MHz 
and has a 50% duty cycle correction too. 

The Information from the sensor is analog source, so it must be discretized for that the FPGA 
can process. For this purpose we have chosen the Analog-Digital Converter (ADC) ADC0820. 
The ADC0820 is an 8 bits resolution converter, it offers a 2^s conversion time and it has a to 
5 Volts analog input voltage range. The element responsible for this task is the ADC control 
component. 

The ADC control component is composed of two modules, the ADC interface module, which 
is a simple finite-state machine (FSM) that implements the communications protocol to acquire 
data of the ADC0820, and the float-point encoder module, which converts the integer value 
into single-precision floating-point format. A block diagram of ADC interface module is 
shown in Fig. 12. 
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Fig. 12. Block Diagram of the ADC control component. 

Now, the information generated by the ADC control component should be processed by the 
corresponding control law. 

The discrete PID controller was synthesized on a FPGA based on equations for the continuous 
PID controller (Ogata, 2010), defined as 



u m = K p (F(t) - F(t)) + K, J (F(t) - F(t))dt + K a 



d{F{t)-F{t)) 
dt 



(10) 
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where iC 



and K d = K„T d . 



An important aspect in the discretization of (10) is the obtaining of a discrete approximation 
of the continuous integral and a discrete approximation of the continuous derivative. 

For discrete approximation of the continuous integral we have used the Adams-Bashforth 
method of the second order (Ascher & Petzold, 1998). This method is defined as 



y[n + 1] = y[n] + -At(3y[n] - y[n - 1]) 



(ID 



Then, if the continuous integral is defined as L (F(t) — F(t))dt, using the Adams-Bashforth 
method, its discrete approximation is defined as 



F[n + 1] = F[n] + -At(3(F[n] - F[n}) - (F[n - 1] - F[n - 1])) 



(12) 



The Fig. 13 shows the proposed architecture for discrete approximation of a continuous 
integral given by (12). 

Stage 3 Stage 4 Stage 5 Stage 6 Stage 7 Stage 8 



f[w]-F[«] 




Fig. 13. Block diagram of the discrete approximation of a continuous integral. 

On the other hand, the discrete approximation of the continuous derivative is obtained based 
on finite differences method of the first order (Burden & Douglas, 2000), using the backward 
difference. This method is defined as 



t^y. _ y[n] - y[n - 1] 

( dt )n ~ At 



(13) 



Then, if the continuous derivative is defined as ^ - K, — —, using the finite differences 
method, its discrete approximation is defined as 

(F[n]-F[n})-(F[n-l}-F[n-l}) 



F'[n] 



At 



(14) 



The Fig. 14 shows the proposed architecture for discrete approximation of a continuous 
derivative given by (14). 

The architecture consists of six multipliers and six adders. Then, it is necessary to implement 
single-precision floating point custom-adder and custom-multiplier. 
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Stage 6 




Fig. 14. Block diagram of the discrete approximation of a continuous derivative. 

The Xilinx ISE Design Suite 12.2 includes the CORE Generator tool, which allows generating 
pre-optimized elements for Xilinx's FPGA. Our controller architecture uses multipliers and 
adders of single-precision floating-point, standard Std-754, generated by this tool. The 
symbols of the multiplier and adder generated by the CORE Generator tool are showed in 
the Fig. 15. 
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Fig. 15. Adder and Multiplier modules generated for Xilinx CORE Generator tool. 

The proposed PID controller architecture is composed of 10 pipeline stages (see Fig. 16) and 
each of them needs 100 cycles to fulfill its function (2 }is), this indicates that the processing 
time of one data is 20 }is (time between 2 consecutive data delivered by the controller to the 
next component, the PWM). The enable signals (Stage_enable0..9) have the control each one 
of the pipeline registers that composed the proposed architecture. 




Fixed-point to 8-bit 

unsigned binary 

conversion 



Stage_enable 



Fig. 16. Architecture proposed for the discrete PID controller implemented into the 
Spartan-3E1600 FPGA. 

In the last stage the PID controller output must be adequacy for the PWM module. This 
adequation consists of Float-point to 8 bit unsigned binary conversion. 
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The last component of the proposed architecture is the PWM. The PWM component consists 
of single up-down counter unit and one magnitude comparator unit (see Fig. 17(a)). 



From GPI 








PWM component 




controller 














A 
B>A 

B<A 
B 
















Up- 


down counter 










> 
















Clk_PWM 








signals 



























_| PWM Top 
1 I I PWM Bottom 




GPI controller 
output value 

/ 



(a) 

Fig. 17. (a) PWM component; (b) PWM outputs. 



(b) 



The PWM output frequency depends of the maximum count value of the counter and the 
Clk_PWM frequency (see figure 17(b)). Then, the PWM output frequency is defined as 



PWM frequency ■■ 



CLK_PWM 



25MHz 
512 



48.828KHz 



(15) 



2(maximuncount + 1) 

The implementation result of the complete architecture for discrete PID controller are reported 
in Table 1. 



Mod. 


Slices 


Flip-Flops 


4-input's 
-LUT's 


Pre-opt -elem. 


Max. Freq 
(MHz) 


1 


5668 (38%) 


8722 (29%) 


8737 (29%) 


1 BRAM (2 %) 
1 DCM (12 %) 


60.37 



Table 1. Discrete PID controller implementation results. 



6. Experimental results 

The PID control and the Pulse Width Modulator (PWM) actuator for the regulation of output 
voltage of the buck converter were implemented in a Spartan 3E board. The only external 
hardware connected to the FPGA for measuring the "buck" converter output voltage was the 
analog digital converter ADC0820. Figure 18 illustrates the block diagram of the FPGA-based 
control system based on PID controller. 



6.1 Requirements of the PID controller 

Figure 19 shows the open-loop response of the "buck" converter with the following 
specifications: L = ImH, C = WOuF, R = 10011, E = 24V, f = 48.828KHz, Av /v = 0.013%, 
Aii = 0.092 and a duty cycle D = 0.75. The output voltage response is a steady-state error of 
5.56% and has a settling time of 15ms. On the other hand, we get that the diagram bode of the 
transfer function given by (4) with the same parameters, has a gain margin Gffl = Inf (at 
Inf rad/sec) and a phase margin Pm = 0.377deg (at 1.58 x 10 4 rad/sec). Given that the 
buck converter system has infinite gain margin, it can withstand greater changes in system 
parameters before becoming unstable in closed loop. Since the system has this characteristic, 
we will design our controllers in closed loop with the following requirements: Overshoot 
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Fig. 18. Block diagram of the FPGA-based control system for PID controller. 

less than 4.32%, Setting time less than 5 milliseconds, Steady-state error less than 1%, and 
Maximum sampling time 40^s. 



20 
18h 

16 
14 
12 
10 



o 
> 



6 

4 




Steady state error 5.56% 



i\ i flpn^mfBnmfynj iMMiiwiiwiif 



-0.01 -0.005 



0.005 0.01 0.015 0.02 0.025 0.03 
Time [s] 



Fig. 19. Output voltage transient response of the "buck" converter with the PID control 
scheme. 

The PID controller gains obtained by the design requirements were: 



K„ = 0.15; T { = 1.2 x 1(T J ; T d = 5.9 x 10 



-3. 



(16) 
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6.2 PID controller into the FPGA 

Figure 20 shows the performance of the PID control law, in the stabilization task for the "buck" 
converter output voltage. As before, we used a constant reference of 18 V. The continuous line 
corresponds to the PID controlled response. The settling time of the response of the "buck" 
converter output voltage through the PID controller, is 13.64 ms. The PID controller tuning 
was done through a third order Hurwitz polynomial. 

Table 2 exhibits the performance of the synthesized controller. The main specifications of the 
transient response, the bandwidth of the PID controller (see Table 2), these frequencies are 
calculated in the closed-loop through the damping ratio and settling time (Messner & Tilbury, 
1999). The damping coefficient value is 0.707, while the value of settling time is: 13.64 ms. 
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Fig. 20. Output voltage transient response of the "buck" converter with the PID control 
scheme. 



Delay time 


h 


2.52 ms 


Rise time 


t r 


4 ms 


Time of peak 


h 


6.24 ms 


Percentage of overshoot 


M p 


2.2 % 


Settling time 


ts 


13.64 ms 


Bandwith 


B w 


414.85 Hz 



Table 2. Specifications of the Controller transient response PID. 

To illustrate the robustness of the PID controller, we made a test with the "buck" converter 
system by suddenly connecting a dynamic load (DC motor) at the output of the "buck" 
converter. Figure 21(a) shows the behavior of the perturbed converter's output voltage and 
the recovery of the output voltage to the desired reference signal when the converter is 
controlled with the PID controller scheme. Also, in Figure 21(b) is shown the u av control 
signal, from the PID scheme implemented in the FPGA. 
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Fig. 21. Output voltage response of the "buck" converter with sudden connection of a DC 
motor. 

7. Conclusions 

In this work, we have applied the Proportional Integral Derivative control scheme, 
synthesized via a Field Programmable Gate Array implementation, for the output voltage 
regulation in a DC /DC power converter of the "buck" type. The performance of the PID 
control action was synthesized via a FPGA. The results obtained by cosimulation allowed 
to study each of the units designed and modeled in VHDL, correcting some errors and, in 
addition, the cosimulation was a perfect tool allowing faster design process to get a full system 
simulation before implement the system in the FPGA board. Also we conclude that the PID 
controller has a good transient response. When we connect a static and a dynamic load to 
the "buck" converter output, we observed that the PID control results in a significantly faster 
response, regarding the output voltage recovery time to the desired reference. Finally, the 
experimental results show the effectiveness of the FPGA realization of both the PID controller, 
in this case, programmed into the FPGA. This methodology of design can be used to design 
switched mode power supplies with efficiency greater than 95%. 
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1. Introduction 

Model predictive control (MPC) is a multivariable feedback control technique used in a wide 
range of practical settings, such as industrial process control, stochastic control in economics, 
automotive and aerospace applications. As they are able to handle hard input and output 
constraints, a system can be controlled near its physical limits, which frequently results in 
performance superior to linear controllers (Maciejowski, 2002), specially for multivariable 
systems. At each sampling instant, predictive controllers solve an optimization problem to 
compute the control action over a finite time horizon. Then, the first of the control actions from 
that horizon is applied to the system. In the next sample time, this policy is repeated, with 
the time horizon shifted one sample forward. The optimization problem takes into account 
estimates of the system output, which are computed with the input-output data up to that 
instant, through a mathematical model. Hence, in MPC applications, a suitable model to 
generate accurate output predictions in a specific horizon is crucial, so that high performance 
closed-loop control is achieved. Actually model development is considered to be, by far, the 
most expensive and time-consuming task in implementing a model predictive controller (Zhu 
& Butoyi, 2002). 

This chapter aims at discussing parameter estimation techniques to generate suitable models 
for predictive controllers. Such a discussion is based on the most noticeable approaches 
in MPC relevant identification literature. The first contribution to be emphasized is that 
these methods are described in a multivariable context. Furthermore, the comparisons 
performed between the presented techniques are pointed as another main contribution, since 
they provide insights into numerical issues and the exactness of each parameter estimation 
approach for predictive control. 

2. System identification for model predictive control 

The dominating approach of the system identification techniques is based on the classical 
prediction error method (PEM) (Ljung, 1999), which is based on one-step ahead predictors. 
Predictive control applications demand models that generate reliable predictions over an 
entire prediction horizon. Therefore, parameters estimated from objective functions based 
on multi-step ahead predictors, generally result in better models for MPC applications (see 
Shook et al. (1991) and Gopaluni et al. (2004) for rigorous arguments). Since the last decade, 
an intense research has been done in order to develop system identification methods focused 
on providing appropriate models for model predictive control. Such methods are denoted 



232 Frontiers in Advanced Control Systems 

as model relevant identification (MRI) in the literature. Strictly speaking, MRI algorithms 
deal with the problem of estimating model parameters by minimizing multi-step objective 
functions. 

Theoretically, if the model structure exactly matches the structure of the actual system, then 
the model estimated from a one-step ahead predictor is equivalent to the maximum likelihood 
estimate, which also provides optimal multi-step ahead predictions. However, in practice, 
even around an operating point, it is not possible to propose a linear model structure that 
exactly matches the system to be identified. Consequently, any estimated model has modeling 
errors associated with the identification algorithm. In these circumstances, models tuned for 
multi-step ahead predictions are more adequate for high closed-loop performance when using 
predictive controllers (Huang & Wang, 1999). In other words, when there is a certain amount 
of bias due to under-modeling (which is the more typical case), the MRI may be considered a 
way of distributing this bias in a frequency range that is less important for control purposes 
(Gopaluni et al., 2003). 

Before formulating the parameter estimation problem in the MRI context, the discrete-time 
linear model structures to be used are specified. 

2.1 Model parameterization 

Consider a linear discrete-time system S with m inputs and p outputs 

y(t) = G (q)u(t) + H (q)e(t) , (1) 

where y(t) is the p-dimensional output column vector at sampling instant t, u(t) is the 
m-dimensional input column vector and e(t) is a p-dimensional zero-mean white noise 
column vector with apxp diagonal covariance matrix R. The system S is characterized 
by the filter matrices Go(q) and Ho(q). The process 1 and the noise models of S are denoted by 
G(q,9) and H(q,9), respectively. In this work, the system model is represented using matrix 
fraction descriptions (MFD) of the form 

G{q,e)=F- l {q)B{q) (2) 

H{q,6) = D-\q)C{q). (3) 

where B(q), C(q), D(q) and F(q) are matrices of polynomials in the shift operator q with 
dimensions pxm, pxp,pxp and p x p, respectively. The parameter vector 9 is composed 
of the coefficients of the polynomials in such matrices. Thus, in order to determine 9, one 
needs to further specify the polynomial matrices in (2) and (3). The matrix B(q) takes the form 



B(q) 



Bn(ij) ••• B lm (q) 



B p i(q) ■■■ B pm (q) 
whose entries are Jlu — 1 degree polynomials 



(4) 



"'/ 



(, ? ) = fo<;y i + ...+fojfV" 



1 Sometimes (Ljung, 1999; Zhu, 2001, e.g.), the process model is referred to as transfer function. 
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for i G { 1, . . . , p } and j 6 { 1, . . . ,m} . One of the simplest choice to parameterize the other 
matrices is through the diagonal form MFD, in which C(q), D(q) and F(q) are diagonal 
polynomial matrices and their nonzero polynomials are all monic, e.g., 

F n ((?) ••• 

F 22 (q) : 



Hi) 



: ■• 

•••0 F pp {q)\ 



(5) 



where the entries of F(q) are v, degree polynomials of the form 

for each i G {1,2,. ..,p}. The diagonal matrices C(ij) and D(q), as well as their respective 
entries, are defined analogously. 

When the diagonal form is adopted, it is possible to decouple the multi-input multi-output 
model into a set of p multi-input single-output (MISO) models in the form 

yi(0 = Ffl'W £ By(q) Uj (t) + §^*l(0 

(6) 



./ = ! 



— / 



m 

y p (f) = F~\q) £ B pj {q) Uj {t) + ^^e p (t) , 
j=i u pp\ c l) 



in which y,- and w,- denote the 2 output and the / input, respectively. 

Unless otherwise stated, it is assumed that all the nonzero polynomials of the matrices have 
the same degree n, that is to say pu = Vj = n, for i G {1, . . . , p} and j G {1, . . . , m}. Although 
this degree is in general not the same as the McMillan degree, this choice considerably 
simplifies the order selection problem and, consequently makes the model structure more 
suitable for applications in large scale processes. 

Besides being simple to understand, the diagonal form has some relevant properties for 
applications in system identification (Zhu, 2001). The main of them is that algorithms 
developed for the SISO (single-input single-output) processes can be directly generalized 
for the multivariable case. Nevertheless, if there are dynamic iterations between different 
outputs, the estimated model based on the diagonal form can present a larger bias error (Lauri 
et al., 2010). Alternatively, one can add elements outside the diagonal of F(q) , not necessarily 
monic polynomials, with the purpose of incorporating the dynamic iteration between the 
process outputs. This approach gives rise to another MFD named "full polynomial form" 
(Ljung, 1999), in which any F(q) entry may be nonzero. This parameterization is also 
employed in one of the identification methods described in Section 3. 

Next, the multi-step objective function used as the basis for the development of the MRI 
algorithms is presented. 
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2.2 The model relevant identification cost function 

Firstly, let us define the p x p filter matrix 

where h(l) is the I impulse response coefficient of H(q,8). 

Thus, the fc-step ahead predictor of the output vector (i.e., the output prediction equation at 
t + k with data available up to instant t) may be expressed as (Ljung, 1999) 

y{t + k\t,9) = W k (q,9)G(q,e)u(t + k) + (I - W k (q,9))y(t + k) . (8) 

According to (8), the fc-step ahead prediction error is 

e(t + k\t,6) = y{t + k) -y(t + k\t,8) 

= W k (q,8)(y(t + k) - G(q,6)u(t + k)) . (9) 

From (7)-(9), note that the fc-step prediction error is related to the one-step through the filter 
matrix 

L k (q,e)±£h(i)q-', (10) 

1=0 

such that 

e(t + k\t) = L k (q,6)e{t + k\t + k-l). (11) 

As argued previously, the main objective of the MRI methods is to provide models that are 
optimized for the generation of predictions over an entire prediction horizon. So, a natural 
choice for the criterion of the parameter estimation problem is the cost function 

P N-k 

Jmm(P,0) = E E l|e(* + *M)Hil/ < 12 ) 

k=\ t=0 

where || ■ H2 denotes the £2 norm. Hence, J m ulti{P>8) quantifies the mean-square error, based 
on predictions ranging from 1 to P steps ahead in a dataset of length N. 

The challenge in estimating the model parameters by minimizing (12) is that such a criterion 
is highly nonlinear in the model parameters. Therefore, suitable optimization algorithms are 
necessary, so that local minima or convergence problems are avoided. Strictly speaking, the 
identification methods to be presented aims at estimating the model parameters based on 
imulti- 

3. Model parameter estimation methods 

In recent years, distinct MRI techniques were proposed based on different principles. One of 
them, conceived by Rossiter & Kouvaritakis (2001), differs from the others since it proposes 
the use of multiple models to generate the predictions. Thus, an optimized model is estimated 
for each fc-step ahead prediction. In spite of providing "optimal" predictions for the entire 
horizon, the number of parameters involved can be quite large, specially for multi-input 
and multi-output processes. It is known that the variance of the parameter estimates is 
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proportional to the ratio between the number of parameters and the dataset length (Ljung, 
1999). Hence, the main drawback of the multi-model approach is the amount of data required 
to estimate a reasonable model set. Such amount of data may be prohibitive in practical 
situations (Gopaluni et al., 2003). Moreover, most of the MPC algorithms are based on a single 
model. For these reasons, the multi-model method is not considered in further analysis. 

In the pioneering work by Shook et al. (1991), the MRI is performed in the context of data 
prefiltering using SISO ARX (Auto Regressive with external input) type models. Huang 
& Wang (1999) extended the previous method, so that a general model structure (e.g., 
Box-Jenkins) could be employed. Some authors, such as (Gopaluni et al., 2003; Lauri et al., 
2010), deal with the parameter estimation problem directly minimizing the MRI cost function, 
using nonlinear optimization techniques. In another approach, proposed by Gopaluni et al. 
(2004), the focus is given to the noise model parameter estimation. In this approach, a 
non-parsimonious process model is estimated, in order to eliminate bias errors (which are 
caused by under-modeling). Then, with a fixed process model, the parameters of the noise 
model are obtained by minimizing the cost function (12). 

In the following subsections, the main MRI techniques are described in more details. 



3.1 The prefiltering approach 

3.1.1 The basic idea 

For the sake of simplicity, the basic idea behind the prefiltering approach is shown using the 
SISO case (m = p = 1). Nevertheless, its worth mentioning that the conclusions directly apply 
to MIMO models represented in the diagonal form MFD. 

In this case, based on predictor (9), the MRI cost function (12) can be rewritten as 



P N-k 



L k {q,e) 



7muM(P,0) = £ E {^f^(y(t + k)-G(c,,e)u(t + k)) 

/c=l t=0 \ n \ c 1> ) 



(13) 



If we introduce an auxiliary variable G(cj, 8) that takes into account the deterministic model 
mismatch, that is 

flfa,0) 4 Gofo) ~ G{q,6), 



then, substituting (1) into (13) gives 
/multi(^0) 



EEY^#S(^''/' (,i '' 1/ -i- / vH-H. 1 (<;)dM-/c)) 



P N-k 



Supposing N — > oo and applying Parseval's relationship to (14) yields 



u(t + k) 
e(t + k) 



(14) 



/multi(^e) 



p 

E 



k=\ 



In J-n 






H(eJ w ,6) 



[G(ei u ',6) H (0] 



R 



H (e-J») 



dw , 
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where <E> M (a>) is the power spectrum of u(t) and <£> eu (io) is the cross-spectrum between e(t) 
and u(t). Now, moving the summation to the inside of the integral, it follows that 



Jmum(P,e) 



1 rn LLl\ L k(e' W ,0)\ r . . . •, 

/ J ^ j -. [2-L G(^,0) H (e^) 



2tt 



|jJ(e/^,0)|" 



<I>„M <5 ra (a;) 
<!>,,, M R 



Ho(e _/w ) 



dto . 



(15) 



From (15) one can see that the deterministic model mismatch is weighted by the input 
spectrum, while the filter 



w muW (^,e) = £ w,(^',e) = — — L I 

£i' ' \H(eJ",0)f 



(16) 



weights the whole expression. But, if P is limited to 1, which implies considering only one-step 
ahead predictions, we obtain 



/multi(-P,0)l , =^" /" 12 ffiC^fl) H, 

lp=i 27T7-7T m^e) L 



(^,0)1^ 
4>«e(w) R 



o(0 x 



G(e^' a ',0) 
H (e-^) 



da; . 



(17) 



Comparing (17) with (15), it is observed that the latter is identical to the first weighted by the 
frequency function 

^multi(^',0)= £k(<>0)| 2 . (18) 

fc=l 
Hence, the estimation of the model parameters by minimizing the MRI cost function (15) is 
equivalent to using standard one-step ahead prediction error estimation algorithms (available 
in software packages, such as Ljung (2007)) after prefiltering the data with (18). As the 
prefiltering affects the model bias distribution and may also remove disturbances of frequency 
ranges that one does not want to include in the modeling, the role of the prefilter may be 
interpreted as a frequency weighting optimized for providing models suitable for multi-step 
ahead predictions. 



3.1.2 Algorithms and implementation issues 

Although the prefiltering artifice is an alternative to solve the problem of parameter estimation 
in the context of MRI, there is a point to be emphasized: the prefilter L mu lti (<?'$) in (18) 
depends on the noise model H(q, 9), which is obviously unknown. 

An iterative procedure called LRPI (Long Range Predictive Identification) to deal with the 
unknown noise model was proposed by (Shook et al., 1991). As mentioned previously, in the 
original formulation only the SISO case based on the ARX structure was concerned. Next, the 
LRPI algorithm is extended to the multivariable case. To this end, the following is adopted 



G(q,6) = A- l {q)B{q) 
H(q,9) = A- 1 (q), 



(19) 
(20) 
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where the polynomial matrix A(q), as well as its entries, are defined analogously to (5). 
According to (19)-(20), the i output equation may be expressed by 



Au(l)yi(t) = '£By(q)uj(t)+e i (t) 



(21) 



Consider the regression cpj(t) 6 R"( m +l) and the parameter 8j 6 ]R"( m +l)^ relative to the i 
system output 



<Pl(t) = [-y,-(t-l),--- ,-yi(t-n),ui(t- !),-■■ ,u m (t-l), 
■ ■ ■ ,U\{t — n) , ■ ■ ■ ,u m (t — n)] 

From (22) and (23), the one-step ahead prediction of y,(f) may be expressed as 

y,(t + l\t,e i ) = cpj(t)8 l . 

Algorithm 1: Extension of the LRPI algorithm to the multivariable case 

Step 1. Set i = 1 (that is, only the first output is considered). 

Step 2. Initialize i^rnulti,! (<?) to 1. 

Step 3. Filter i/,(f) and each input uAt) for/' 6 {1, . . . ,m\ with L mu i^j(q), i.e. 



]/, (0 — ^multi,i 



U f(t) 4 



(?)y/(0 

i-multi,i( £ ?) 









"(f) 



i-multi,i('?). 
Step 4. Based on (25)-(26), construct the regression vector analogously to (22), so that 

f{(t)=[-y{{t-\),---,-y f l {t-n),uf T (t-\),---,uf T {t-l)\ T . 

Step 5. Estimate the parameter vector 0, by solving the linear least-squares problem 

i = argrrunWy i (f)-<pf(f)0; N 

0; t y 



(22) 
(23) 

(24) 



(25) 



(26) 



(27) 



(28) 



Step 6. Update L mu itv(<7) through (10) and (18), based on the noise model A,, (q) estimated 
in the previous step. 

Step 7. Continue if convergence of 6, occurs, otherwise go back to Step 3. 
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Step 8. If i 7^ p, go back to Step 2, with i = i + 1. Otherwise, concatenate the estimated models 
into a MIMO representation. 

Remarks: 

• For the multi-output case, there are p different filters L mu lti(<?!h each one associated with 
the i output and denoted by L mu itL,-((j). 

• With respect to Step 6, as L mu \ ti (q) is a spectral factor of £- mu i ti (e-' a '), spectral factorization 
routines, such as the one proposed in Jezek & Kucera (1985), can be used for solving (18). 

• A natural choice to determine the convergence of the algorithm is to check if the £2 norm 
of the difference between the parameter estimates in two consecutive iterations is less than 
S. Experience has shown that a reasonable value for S is 10 . 

Alternatively, instead of using an iterative procedure as previously, in the method proposed 
by Huang & Wang (1999) named MPEM (Multi-step Prediction Error Method), a fixed noise 
model estimate is employed in order to get L mu \ t [(q). In what follows, the multi-step 
prediction error algorithm is described, based on the MFD parameterized by (2)-(5). 

Algorithm 2: MPEM algorithm based on the diagonal form matrix fraction description 
Stepl. Seti = 1. 

Step 2. Get initial estimates of Cu(q), Da(q), Fu(q) and, for j e {l,...,m}, Bij(q), using 
standard prediction error methods, namely, based on a one-step ahead cost function 
(17). 

Step 3. Use a spectral factorization routine to solve (18), in which the filters defined in (10) are 
calculated through the impulse response of the estimated noise model UJ, (q)Ca(q). 

Step 4. Filter i/,(f) and each input Uj(t), j € {1, . . . , m), with CjT 1 (q)Djj(q)L mu i ti j(q). 

Step 5. With the fixed noise model D^ (q)Cjj(q), calculated in Step 2, estimate Bfi(q),..., 
Bi m (q), Fa(q) by minimizing the output-error cost function 

V oe j(Bn(q),...,B im (q),F ii (q)) = £l/ i (t)-F-J(q)£B v (q)uj(t)\ . (29) 

Step 6. If i 7^ p, go back to Step 2, with i = i + 1. Otherwise, concatenate the estimated models 
into a multi-output representation. 

Remarks: 

• Once more the diagonal form MFD property, which allows the independent treatment 
of each model output, is applied to extend the parameter estimation algorithm to the 
multivariable framework. 

• The prefilters of Step 2 differ from the ones used in the LRPI algorithm by the additional 
terms CJ, (q)Djj(q), each one for i G {1, . . . , p}, which represents the inverse of the i 
output noise model. Hence, while the filters L mu i t j , (q) aim at providing optimal weighting 
for multi-step predictions, the additional terms intend to remove the noise influence for 
models represented as (6). 
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• The minimization of (12) is replaced by two nonlinear optimization problems in the MPEM 
algorithm. At first, it might seem that there is no relevant advantage in such an approach. 
Nevertheless, it is important to say that the MISO Box-Jenkins identification from Step 2, 
as well as the minimization of the output-error cost function in (29), can be performed 
using available software packages (Ljung, 2007, e.g.). Moreover, for models parameterized 
as (2)-(5), the numerical complexity of these problems are considered to be lower than the 
one of minimizing / mu iti directly. 

The LRPI algorithm involves only linear least-squares problems, which have many 
advantages. The most important one being that (28) can be solved efficiently and 
unambiguously (Ljung, 1999). The price paid for a simple parameter estimation algorithm 
is the adoption of a limited noise model structure. Consequently, the estimate of the H(q,6) 
entries may be inaccurate, which affects the calculation of each filter L mu i ti ;((?)• In turn, MPEM 
considers a more flexible noise model structure. However, local minima or convergence issues 
due to nonlinear optimization methods in Steps 2 and 5 may degrade the quality of the 
estimates. Therefore, the MPEM should outperform the LRPI algorithm, provided that the 
global minimum is achieved in the estimation steps. Anyway, it is suggested that models are 
estimated using more than one method and select the one which yields the best multi-step 
ahead predictions. 



3.2 Direct optimization of the cost function 

In the prefilter approach described previously, the filters L mu itv(<7) are calculated using 
any spectral factorization routine. Hence, as these filters are approximations of (18), the 
identified model ability to generate multi-step ahead predictions depends on the degree of 
the approximation and on the accuracy of the disturbance model estimate. But there is no 
need to worry about these aspects if the MRI cost function (12) is minimized directly. On the 
other hand, the model parameterization should be chosen carefully, to minimize numerical 
problems in the nonlinear optimization algorithm. In Lauri et al. (2010) a "full-polynomial 
form" ARX model 

A(q)y(t) = B(q)u(t) + e(t), (30) 



with 



A(q) 



A U {q) ■■■ A lp {q) 
_A pl (q) ■■■ A pp (q)_ 



I + AWq- 1 + ... + A < - n '>q- n , 



(31) 



whose entries are 



A,j(q) 



1 + afq- 1 + ... + «JV" , for i = j 
a ij ( ?~ 1 + • • • + a u 1~ n • otherwise 



and the polynomial matrix B(q) is defined as in (4). 



1 Note that, in order to consider output interaction, the polynomial matrix A(q) is not restricted to heing 
diagonal, as in the LRPI algorithm. 
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G R 



n(m+p)x-p 



For this model structure, let us introduce the parameter matrix 

a (1) ■■■ fl (1) ■■■ a (n) ■■■ a (n) b (l) ■■■ fo (1) ••• b (n) ••• 6 (n) ' 

i m ... a m ■■■ fl (n) ■■■ fl (n) 'fe (1) ■■■ fe (1) ■■■ fo (n) ■■■ b {n) 

v\ ' > u pp ' ' p\ ' ' PP ' v\ ' > u pmr ' p\ ' i u pm 

(32) 
and a particular regression vector denoted by <p{t + k\t, 0) £ ]R"( m +P) / which is composed of 
inputs up to instant f + k, output data up to t and output estimates from t + ltot + k— 1, for 
instance 

f{t + 2\t,@) = \-f(t + l\t,0),-y T (t),--- ,-y T {t-n + 2),u T (t + !),-■■ ,M T (f-n + 2)| 

and for an arbitrary k 

cp{t + k\t,&) = \-y T (t + k-l\t),--- , -y T (t + k - n\t),u T {t + k - 1), ■ ■ ■ ,u T (t + k-n)] , 

(33) 



where 



y ^ ' ' \ y(s) , otherwise. 



From (32) and (33), the fc-step ahead prediction of y(t) is given by 

y(t + k\t,@) = e T f(t + k\t,&). (34) 

Although the predictor y(t + k\t, 0) is nonlinear in the parameters, it is important to notice 
that it can be calculated recursively, from y(t + l\t) fork 6 {2,...,P} using (34). This is the 
main reason why the ARX structure was adopted. For another thing, if a more flexible model 
structure is adopted, the fc-step ahead predictor equation would be much more complex. 

Thus, based on the MRI cost function (12), the parameter estimation can be stated as a 
nonlinear least-squares problem 

P N-k 
© = argmin£ £ \\y{t) - ® T W + Mt,©)\\ 2 2 , (35) 

© fc=l f=0 

which must be solved numerically. The Levenberg-Marquart algorithm is used in Lauri et al. 
(2010) in order to minimize (35). 

3.3 Optimization of the noise model 

In Gopaluni et al. (2004) it is shown that, in the absence of a noise model, there is no significant 
difference between MRI and one-step ahead prediction error methods. On the other hand, 
when the signal to noise ratio is small, the one-step ahead predictors yield worse results 
for P-step ahead predictions than MRI methods. Thus, in these circumstances, a suitable 
disturbance model is crucial to generate accurate multi-step ahead predictions. 

Any identified model has bias and variance errors associated with the identification algorithm. 
While the former is typically associated to model mismatch (such a mismatch can be either in 
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the process model or in the noise model) and the second is due to the effect of unmeasured 
disturbances. If that there is no significant cross-correlation between the noise and system 
input (in open-loop, e.g.), the bias errors in the process model may be eliminated by using 
high order FIR (Finite Impulse Response) models (Zhu, 2001). Under that assumption, the 
modeling errors are restricted to the noise model. 

With this in mind, in Gopaluni et al. (2004) the authors propose a two-step MRI algorithm in 
which the process is represented by a FIR structure, with sufficiently high order so that bias 
errors due to the process model can be disregarded. Then, the noise model parameters are 
estimated using a multi-step cost function. 

Consider the multivariable FIR model 

G ¥m (q,6) = B(q) (36) 

where the polynomial matrix B(q) is defined as in (4). The noise model H(q,rj) is 
parameterized using diagonal MFD. These choices are equivalent to (6) with F(q) = I. As 
the estimation of G(q) and H(q) are performed separately, in this subsection, the parameter 
vector is split into two parts, such that the noise model parameter vector is explicitly referred 
to as rj. So, the i output noise model structure is 

Let us introduce the residual of the process model relative to the i output 

m 

w<(0 4 y.-(0 -£%(?)«/(*)• (38) 

Then, based on (7)-(9), the fc-step ahead prediction error of the i output can be written as 
e,(f + k\t,9) = y t {t + k)- y t {t + k\t,6) 

L h i(i)r l )°Mvi(t)- 09) 



(=0 



Cu(q) 



As Ca(q) and Djj(q) are monic polynomials, the impulse response leading coefficient /j,(0) is 
always 1. With this, expanding (39) yields 

£ ,(f + k\t,0) = v t (t + k)+ hi{l)vi(t + k- 1) + ... + h,{k - l)v,{t + 1) 
-cjpe^t + k - 1| t, &) - . . . - c^tiit + k- Oi\t,e) 
+d\l ) L kA (q)v i (t + k-l) + ... + dfh^Viit + k - B { ) . (40) 



3 Part of the notation introduced in Section 2.2 is particularized here to the single-output context. For 
instance, /;,(/) and L^iiq) are the equivalent to the ones defined in (10), but related to the ! output. 
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For a compact notation, we define 

7W=[M1) h,(k-l),l4 ] cf 1 ',^ iff (41) 

9v(t>Vv)- [-v t (t + k-l),...,-v t (t + l),y t (t + k)-v,(t + k),e,(t + k-l\t,9) r 

...,E l (t + k- l i l \t,e),-Ut + k-l),--- ,-Ut + k-ji,)] 7 (42) 

where 

£i(t + k)±F k/i (q)vi(t + k). 

Then, we can rewrite (39) as 

£i(t + k\t) = y t (t + k\t) - q>li(t,?] kt i)Ti k/i . (43) 

In light of the aforementioned paragraphs, the MRI algorithm that optimizes the noise model 
is summarized as follows. 

Algorithm 3: MRI with optimized noise model 

Stepl. Seti = 1. 

Step 2. Fix an a priori noise model to the i output, for instance 

Q,-(?) = 

Dii(q) 
and estimate a multi-input single-output high order FIR model using standard PEM. 
Step 3. With the estimate Gfir(<?)/ from the previous step, solve the optimization problem 

N-P P 2 

Vp,i = argmin £ £ [y,{t + k\t) - q> k>i (t,r]k,i)Vk,i) ( 44 ) 

Ipa (=1 fc=a 

subject to 

/;,(/) = hi(l) (tji), for any I = {1,2,...,? - 1} (45) 

where hj{l) (j/;) indicates the Z impulse response coefficient of (37), which is obtained 
by polynomial long division of C,,(q) by D#(<j). 

Step 4. If i t^ p, go back to Step 2, with i = i + 1. Otherwise concatenate the estimated models 
into a single MIMO representation. 

Remarks: 

• Besides providing unbiased estimates under open-loop conditions, FIR models are suitable 
in this case because the parameters of Gfir can be efficiently estimated using linear 
least-squares. 

• A numerical optimization method is required to solve the parameter estimation problem. 
Nevertheless, the Levenberg-Marquart algorithm mentioned in the previous subsection 
can not deal with constraints. One of the nonlinear optimization algorithm possibilities 
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is the Sequential Quadratic Programming (SQP), which can handle nonlinear constraints 
such as (45). In Gopaluni et al. (2004) it is shown that if a noise model of the form 

l q 

is adopted, then the constraint (45) may be expressed through a linear in the parameters 
equation. In this case, (44) can be solved using the standard Quadratic Programming (QP) 
method. 

4. Simulations 

The main features of the aforementioned MRI techniques are analyzed using two simulated 
examples. At first, a SISO process is considered in order to illustrate the influence 
of the prediction horizon length P in the modeling errors presented by the identified 
models. Moreover, the performance of each technique is evaluated based on datasets with 
distinct signal-to-noise ratios (SNR). After that, the closed-loop performance provided by the 
estimated models is assessed. To this end, the Quadratic Dynamic Matrix Controller (QDMC) 
(Camacho & Bordons, 2004) and a multivariable distillation column benchmark (Cott, 1995a;b) 
are employed. 

4.1 SISO process example 

Consider the third-order overdamped system proposed in Clarke et al. (1987) 

r 0.00768<?- 1 + 0.02123<T 2 + 0.00357^- 3 

° {q) ~ 1 - 1.9031<r! + 1.1514^-2 - 0.215&T 3 ' ( ' 

with a random-walk disturbance, that is 

1 q 

The process is excited in open-loop by a Pseudo Random Binary Sequence (PRBS) switching 
between [—0.1, 0.1] with a clock period of 5 times the sampling interval. The noise variance is 
adjusted such that the signal-to-noise ratio (SNR) is 3 (in variance). A record of 1200 samples 
is collected, which is shown in Fig. 1. The dataset is split into two halves: the first is used for 
estimation and the second one for validation purposes. 

The following reduced-complexity model structure is assumed 4 

G(^)= ^; + y (48) 

1 + ciiq 1 

1 + c^q- 1 
H(q,9) = ^*— . (49) 



'11 



4 Except for the noise model optimization method (Subsection 3.3), in which d n is fixed to — 1, so that 
parameter estimation can be handled using standard quadratic programming. 
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Fig. 1. The dataset from the third-order process (46)-(47). 

Before analyzing the capacity of the estimated models to generate accurate multi-step ahead 
predictions, it is worth noting the influence of the prediction horizon length. The magnitudes 
of L mVi \ ti (e1 w ), for P = {1,2,5,10,15}, are shown in Fig. 2. As can be seen, L mu m(q) is 
a low-pass filter, whose cut-off frequency decreases as P increases. Such behavior occurs 
whenever the disturbance spectrum is concentrated on low frequencies (Gopaluni et al., 2003). 
Hence, according to (15), the higher the prediction horizon length, the narrower the error 
weighting. 

As a consequence, an increase in P leads to lower modeling errors in low frequencies, but the 
frequency response of the estimated models are away from the actual one at high frequencies. 
This behavior is depicted in Fig. 3, which presents the absolute value of the difference between 
the actual and the estimated (from models obtained using the MPEM algorithm) frequency 
responses. One can also notice that the effect of increasing P is more prominent in the range 
[1, 5] than between [5, 15] . Furthermore, as shown in Farina & Piroddi (2011), for sufficiently 
high values of the prediction horizon length, models estimated based on multi-step prediction 
errors converge to the output (simulation) error estimate. 

The cost function / mu iti, defined in (12), is applied to quantify the model accuracy in terms 
of multi-step ahead predictions. It is emphasized that such accuracy is quantified using fresh 
data, that is to say, a distinct dataset from the one used for estimation purposes. In what 
follows, the performance of the MRI techniques are investigated using two sets of Monte 
Carlo simulations, each one with 100 distinct white-noise realizations. In order to visualize 
the SNR effect on different parameter estimation methods, in the first simulation set, the SNR 
is maintained in 3 and in the other one it is increased to 10. The histograms of / mu iti for the 
methods described in Section 3 are depicted in the rows of Fig. 4, for P = 8. The left and the 
right columns present the results for the signal-to-noise ratios of 3 and 10, respectively. The 
main Monte Carlo simulation results are summarized in Table 1, which reports the mean and 
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the standard deviation of ] mu m- For comparison, the last column of Table 1 also presents the 
results produced by the standard (one-step ahead) PEM, based on a Box-Jenkins structure. 

The histograms in Fig. 4, as well as Table 1, show that the MPEM and the noise model 
optimization algorithms presented the smallest / mu iti (that is, the most accurate multi-step 



E -10- 




Frequency (rad/s) 
Fig. 2. Magnitude frequency response of i mu lti(^ a; ) ror increasing prediction horizon length. 




10" 
Frequency (rad/s) 



Fig. 3. Absolute value of the modeling error in the frequency domain, for models estimated 
with P= {1,2,5,10,15}. 
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Fig. 4. Histograms of / mu iti for each MRI method. 

predictions) with the lowest variance, which means that these methods are less sensitive 
to a particular realization. On the other hand, LRPI and direct optimization showed worse 
performances because these methods are based on the ARX model structure, which is quite 
different from the process (46)-(47). Another aspect that may be noticed is that, as expected, 
a higher SNR leads to a smaller / mu i t ; mean (more accurate models are expected) and lower 
deviations of the estimates. 

Actually, the performances of the methods based on ARX structure may be interpreted in a 
broader sense. Although in MRI the effect of bias due to model mismatch is reduced in the 





SNR 


LRPI 


MPEM 


Direct 
optim. 


Noise model 
optimization 


Standard PLM 
(Box-Jenkins) 


mean(/ mu i ti ) 


3 


0.1526 


0.0111 


0.0786 


0.0178 


0.0209 


10 


0.1218 


0.0074 


0.0496 


0.0056 


0.0172 


std(/ multi ) 


3 


0.0536 


0.0045 


0.0239 


0.0163 


0.0042 


10 


0.0668 


0.0015 


0.0104 


0.0049 


0.0014 



Table 1. Mean and standard deviation of the cost function. 
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parameter estimation step, the task of selecting a suitable model structure is still crucial to the 
success of a system identification procedure. This statement is also supported by the fact that, 
according to Table 1, considering a more favorable structure and the one-step ahead PEM 
is more effective than an inadequate structure whose parameters are estimated based on a 
multi-step cost function. 

4.2 Multivariable system example 

The Shell benchmark process is a model of a two-input two-output distillation column (Cott, 
1995a;b). The inputs are overhead vapour flow and reboiler duty, denoted here as U\ and u 2 , 
respectively. The outputs are the column pressure 



Ayi(f) 



0.6096 + 0.4022a- 1 a . . 
= ^-Ami (t) + 



1 - 1.5298CJ- 1 + 0.574(7 

A 

+ 



0.1055 - 0.0918a- 1 A . , 
1 ~Au 2 (t) 



1 -1.5298,- 1 +0.574,- 



^l(') 



1 - 1.5945,- 1 + 0.5945,- 2 
and the product impurity 

5 x 10 5 
^ = 0m65 u 2 (t-7)-1500 + ° m35 ^ - X > I - 1.6595,- + 0.6595, 



A 
1.6595,- 1 - 



=2*2(0 



(50) 



(51) 



where Ay\, Aui and An 2 are deviation variables around the nominal operating point 
(specified in Table 2), that is 

Ayi(f) =yi(t)-yi 

Aiii(t) = U\(t) — Ui 
Au 2 (t) = u 2 (t) — u 2 . 



Variable 


Nominal setpoints 


Normal operation 


Pressure (y ± ) 


2800 


2700 < yi < 2900 


Composition (y 2 ) 


500 


250 <y 2 < 1000 


Overhead vapour flow (u\) 


20 


10 < U\ < 30 


Reboiler flow (u 2 ) 


2500 


2000 < u 2 < 3000 



Table 2. Summary of distillation column operating conditions. 

The disturbances are generated using uncorrelated zero-mean white noises e\ and e 2 , such 
that std(ej) = 1.231 and std(e 2 ) = 0.667. The parameter A is set to 0.2. The Shell 
benchmark is widely used to evaluate multivariable system identification or model predictive 
control strategies (Amjad & Al-Duwaish, 2003; Cott, 1995b; Zhu, 1998, e.g.). Besides being 
multivariable, the model (50)-(51) offers additional complications: as the disturbances are 
nonstationary, one of the outputs (product impurity) is slightly nonlinear and the overhead 
flow (u\) does not affect the impurity level (y 2 )- 



For more details about the simulator operating conditions, the reader is referred to (Cott, 1995b). 
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The process is excited in open-loop using two uncorrelated random binary sequences (RBS), 
with u\ varying from [15,25] and «2 from [2400,2600]. The minimum switching time of U\ 
and U2 is 12 and 6, respectively. The dataset is comprised of 1600 samples, where the first half 
is used for estimation (see Fig. 5) and the rest for validation. 

The elements of the transfer function matrix G(q,9) and of the noise models are first order. 
Initially, the input delay matrix 

" 0' 



"k 



37 7 



(52) 



was estimated applying the function delayest of the Matlab™System Identification toolbox 
(Ljung, 2007). Notice that except for the entry in which there is no coupling (u\ —¥ y-i), the 
values in W; c coincide with the actual input delays. Thus, before proceeding to the parameter 
estimation, the input sequences are shifted according to Hj.. 




800 



200 400 600 800 

Samples 




200 400 600 800 
Samples 



Fig. 5. Estimation dataset from the Shell benchmark simulation. 

The models estimated with P = 40 are evaluated based on the multi-step prediction errors (12) 
using the validation dataset, which are presented in Table 3. The most accurate multi-step 
predictions are generated by the MPEM and the 1-step ahead PEM. This is because, as in 
the SISO example, the Box-Jenkins structure employed by both methods best suits the process 
dynamic behavior. Another relevant point is that the noise model optimization yields unstable 





Output 


LRPI 


MPEM 


Direct 
optim. 


Noise model 
optimization 


1-step PEM 
(Box-Jenkins) 


/multi x 10 4 


1 


0.1154 


0.0328 


0.1475 


CO 


0.0322 


2 


3.2887 


2.5072 


3.6831 


CO 


2.5294 



Table 3. Multi-step prediction error. 
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predictors (due to zeros outside the unitary circle). Consequently, the sum of the prediction 
errors tends to infinity. 

The standard PEM provided multi-step predictions as accurate as the MPEM, even for a 
sub-parameterized model, which is the case of this example. This result suggests that the 
under-modeling issue is not the most prominent for this situation. In addition, the fact that 
the disturbance intensity is very high, besides being concentrated on the low frequencies 
(the same frequency range that should be weighted to attain improved multi-step ahead 
predictions) disfavors the MRI approach. 

In order to test the robustness of the methods to input-output delay estimation errors, a new 
estimation is carried out with a modified delay matrix nf , in which the dead-time from Ui to 
1/2 is changed from 7 to 8 samples. As shown in Table 4, the MRI methods are less sensitive to 
this parameter than the 1-step ahead PEM. 

1-step PEM 



Jmulti x 10 



Output 



LRPI 



MPEM 



3.1669 2.4854 



Direct 
optim. 



3.8126 



Noise model 
optimization 



(Box-Jenkins) 



3.0794 



Table 4. Multi-step prediction error of the 2 output when there is a slight mismatch in one 
of the input delay matrix element. 

At this point, the performance of the estimated models is investigated when they are 
employed in a QDMC controller (Camacho & Bordons, 2004) with a prediction and control 
horizons of 40 and 5, respectively. The output Q and the manipulated 1Z weighting matrices 
are (Amjad & Al-Duwaish, 2003) 



Q 



1 

02 



and 1Z 



20 
02 



The closed-loop responses using the QDMC controller when each set-point is excited with a 
step of amplitude 1% of the nominal output values are presented in Fig. 6 and 7, where the 
first one is related to the input delay matrix Hj. in (52) and the other refers to nt. The results of 
the closed-loop validation are also summarized in Table 5, which shows the integrated square 
error (ISE) for each controlled variable: t/i and 1/2- 

In a general way, the first output is closer to the set-point than 1/2- This may be explained 
by the intensity of the disturbance introduced in each output, by the fact that the plant is 
non-linear whereas the identified models are linear and, finally, due to the presence of a zero 
in the transfer matrix which consequently affects the quality of the estimated model. 

From Fig. 6, one can notice that all the controllers achieved similar responses for the 
column pressure (1/1). Concerning the other output (product purity), the closed-loop behavior 
provided by the standard PEM and the MPEM are very close (accordingly to multi-step 
prediction errors depicted in Table 3). Analogously, the LRPI method yielded a better 
performance than the direct optimization. Besides, as these two methods showed a worse 
multi-step prediction accuracy, it reflected in the MPC performance. 

As shown in Fig. 7 and according to Table 5, the prediction capacity deterioration of the 
one-step ahead PEM, due to the delay matrix modification from nj. to nt also leads to a 
worse closed-loop response. On the other hand, the closed-loop performances provided by 
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the models estimated through MRI algorithms are less sensitive to errors in the time delay 
determination. 



S 2830- 



- LRPI 

MPEM 

2820 Direct optim. 

1-step PEM 

■ ■ Set-point 



2810- 
2800- 



50 



V~^ 



100 



150 



200 250 300 
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150 200 

Samples 



300 



Fig. 6. Closed-loop response based on an accurate input delay estimation. 
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Fig. 7. Closed-loop response for a mismatch in one of the input-output delay matrix entry. 
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Output 


LRPI 


MPEM 


Direct 
optim. 


1-step l J hM 
(Box-Jenkins) 


ISE xlO 3 : n k 


1 


2.1921 


2.0777 


2.4091 


2.0711 


2 


0.3618 


0.3182 


0.3747 


0.3171 


ISE xlO 3 : n* k 


1 


2.1689 


2.0883 


2.4081 


2.2016 


2 


0.3519 


0.3119 


0.3802 


0.3667 



Table 5. Integrated square error (ISE) of the controlled variables. 

5. Conclusions 

This chapter focused on parameter estimation algorithms to generate suitable models for 
predictive controllers. The branch of identification known as MRI was studied and several 
different ways to obtain models were presented. They must be estimated having in mind that 
they must be accurate to predict multi-step ahead. Some of these techniques were published 
considering just the single-input single-output case and in this work they were extended 
to the multivariable framework. In order to compare the different algorithms, they were 
implemented and tested, employing a SISO and a MIMO plant. In the comparisons, the 
standard PEM (built to provide optimal one-step ahead predictions) was also included. 

In the analysis with the SISO process, the long range prediction capacity of some of the MRI 
methods (MPEM and noise model optimization) was superior to the results generated by 
the standard PEM, based on a Box-Jenkins structure. In addition, the influence of the model 
structure was also highlighted in a model relevant identification context, since the standard 
PEM (with a Box-Jenkins) produced more accurate multi-step ahead predictions than the LRPI 
and the direct optimization algorithms, which are based on a less flexible model structure. 

The tests performed with the multivariable plant were more concerned about the use of the 
MRI and PEM models, when applied to a predictive controller. The results obtained were 
not so convincing about the advantages of using multi-step prediction based methods in the 
predictive controller design, since the one-step PEM (with a Box-Jenkins model), even with 
structure mismatch, provided results that were comparable to the best ones obtained with the 
model relevant identification methods. However, it was also shown that when there was a 
slight error in the evaluation of the time delay of one of the input-output pairs, the advantage 
of the MRI approach became evident. 

Although the excitation signal design and the model structure selection are beyond the scope 
of this work, the examples presented the complete system identification procedure, from the 
input signal generation, going through the use of different algorithms to estimate the model 
parameters up to the validation of the models through the verification of their prediction 
capacity. Besides, the obtained models were applied to a predictive controller to evaluate 
their performance in controlling a multivariable process. 

The system identification for MPC is a subject prone to further research. The effect of 
multi-step prediction error methods on the closed-loop performance needs to be further 
investigated. Another important theme to be studied is in which situations the use of MRI 
methods for developing models for predictive controllers is in fact advantageous as compared 
to classical prediction error methods. 
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1. Introduction 



Models are extensively used in the design and implementation of advanced process control 
systems. In model predictive control (MPC), model of the plant is used to predict the future 
output of the plant using the current and future optimal inputs and past outputs. Therefore, 
the design of MPC, essentially, includes the development of an effective plant model that can 
be used for predicting the future output of the plant with good accuracy (Camacho & Bordon, 
2004; Rawlings, 2000). Models can be developed either from purely theoretical analysis 
(conservation principles, thermodynamics, etc.) or from experimental data or somewhere in 
between. The process of model development from experimental data is known as system 
identification. The identification test can be conducted either in open-loop (open-loop 
identification) or while the plant is under feedback control (closed-loop identification). 

The theory of linear system identification is well developed and there are already numerous 
literatures. The pioneering work in system identification was done by Ljung (1999) and his 
book provides detailed theoretical foundation for system identification. The book by Nelles 
(2001) is also a very practical book and highly recommended for practitioners both on linear 
and non-liear system identification. Heuberger, et ah, (2005) authored a very comprehensive 
book on modeling and identification using rational orthonormal basis functions, though 
current developments in application of OBF for MPC, closed-loop identification, etc., were 
not included. 

There are several linear dynamic model structures that are commonly used in control 
relevant problems. They have two general forms, i.e., the state space and input-output 
forms. In this chapter, we deal with the latter form also called transfer function. The most 
common linear input-output model structures can be derived from one general structure (1). 
The general linear structure consists of a deterministic component, i.e., the plant input , u(k), 
filtered by a linear filter and a noise component, i.e., a white noise, e(k), filtered by a 
corresponding linear filter. 

y{k)= _W_ u{k) + _CW_ e{k) fi) 

F(a)A(a) D{a)A{a) 
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The q in (1) is the forward shift operator defined as q u(t) = u(t + 1) and q^ 1 is the delay 
(backward shift) operator, q~ x u{t) = u(t - 1). 

The various commonly used structures can be easily derived from the general model 
structure by making some assumptions. The ARX model can be derived from (1) by 
assuming F(q) = D(q) = C(q)= 1. Therefore, the ARX model structure has the form 

V (k) = ^\u(k) + ^-e(k) (2) 

A(q) A{q) 

The Auto Regressive Moving Average with Exogenous Input (ARMAX) can be derived from 
(1) by assuming F(q) = D(q) = 1. 

y { k) = ^\u ( k) + ^le { k) (3) 

Other linear model structures are listed below: 
Box Jenkins (BJ): 

y{k) = Mu {k ) + ^l e{ k) (4) 

F(q) D(q) 



Output Error (OE): 



y(k) = ^-u(k) + e (k) (5) 



Finite Impulse Response (FIR): 

y(k) = B(q)u(k) + e(k) (6) 

It should be noted that in FIR model structures the filters are simple delays. Equation (6) can 
be expanded into 

y{k) = (b.q- 1 + b 2 q- 2 + ... + b m <f >(fc) + e(k) (7) 

The selection of the appropriate model structure for a specific purpose, among other factors, 
depends on the consistency of the model parameters, the number of parameters required to 
describe a system with acceptable degree of accuracy and the computational load in 
estimating the model parameters. The optimality of model parameters is generally related to 
the bias and consistency of the model. Bias is the systematic deviation of the model 
parameters from their optimal value and inconsistency refers to the fact that the bias does 
not approach zero as the number of data points approach infinity (Nelles, 2001). The most 
widely used linear models are Step Response, ARX and FIR models (Ljung, 1999; Nelles, 
2001). Their popularity is due to the simplicity in estimating the model parameters using 
the popular linear least square method. However, it is known that all of these three model 
structures have serious drawbacks in application. The ARX model structure leads to 
inconsistent parameters for most open-loop identification problems and the FIR and step 
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response model need very large number of parameters to capture the dynamics of a 
system with acceptable accuracy. The inconsistency in the ARX and also in ARMAX 
model parameters is caused by the assumption of common denominator dynamics for 
both the input and noise transfer functions given by 1/A(q), which implies that the plant 
model and the noise model are correlated. In reality, this is rarely the case for open loop 
identification problems. The output error (OE) and the Box Jenkins (BJ) model structures 
assume independent transfer function and noise models, and hence they allow consistent 
parameter estimation. However, determination of the model parameters in both cases 
involves nonlinear optimization. In addition, in case of BJ, because of the large number of 
parameters involved in the equation, it is rarely used in practice, especially, in MIMO 
systems. 

Orthonormal Basis Filter (OBF) models have several advantages over the conventional 
linear models. They are consistent in parameters for most practical open-loop systems and 
the recently developed ARX-OBF and OBF- ARMAX structures lead to consistent parameters 
for closed loop identification also. They require relatively a fewer numbers of parameters to 
capture the dynamics of linear systems (parsimonious in parameters) and the model 
parameters can be easily estimated using linear least square method (Heuberger, et al., 2005; 
Heuberger, et al., 1995; Ninness & Gustafsson, 1997; Van den Hof, et al, 1995). MIMO 
systems can be easily handled using OBF and OBF based structures. In addition, recent 
works by Lemma and Ramasamy (Lemma & Ramasamy, 2011) prove that OBF based 
structures show superior performance for multi-step ahead prediction of systems with 
uncertain time delays compared to most conventional model structures. 

Among the earliest works on rational orthonormal bases was contributed by Takenaka 
(1925) in the 1920's in relation to approximation via interpolation, with the subsequent 
implications for generalized quadrature formula. In subsequent works, in the 1960s, Walsh 
(1975) contributed extensively in the applications of orthonormal bases for approximation, 
both in discrete time and continuous time analysis. In similar periods, Wiener (Wiener, 
1949) examined applications of continuous time Laguerre networks for the purpose of 
building optimal predictor. Van den Hof, et al., (1995) introduced the generalized 
orthonormal basis filters. They showed that pulse, Laguerre and Kautz filters can be 
generated from inner functions and their minimal balanced realizations. Ninness and 
Gustafsson (1997) unified the construction of orthonormal basis filters. Lemma, et al., (2011) 
proposed an improved method for development of OBF models where the poles and time 
delays of the system can be estimated and used to develop a parsimonious OBF model. On 
another work (Lemma, et al., 2010) it was shown that BJ type OBF models can be easily 
developed by combing structures with AR and ARMA noise model. Some works on closed- 
loop identification using OBF based structures have also been presented (Badwe, et al., 2011; 
Gaspar, et al., 1999; Lemma, et al., 2009; Lemma & Ramasamy, 2011). 

2. Development of conventional OBF models 

Consider a discrete time linear system 

y{k) = C(q)u{k) (8) 
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where G(q) = transfer function of the system. A stable system, G(q), can be approximately 
represented by a finite-length generalized Fourier series expansion as: 

g(<?) = &/*(<?) (9) 

1=1 

where {/,}, i =1, 2, ..., n are the model parameters, n is the number of parameters, and fi(q) 
are the orthonormal basis filters for the system G(q). Orthonormal basis functions can be 
considered a generalization of the finite length fourier series expansion. Two filters f\ and fi 
are said to be orthonormal if they satisfy the properties: 

(/i(<7),/2(<7)> = (10) 

||/i(<7)|H/2(<7)| = l (11) 

2.1 Common orthonormal basis filters 

There are several orthonormal basis filters that can be used for development of linear OBF 
models. The selection of the appropriate type of filter depends on the dynamic behaviour of 
the system to be modelled. 

Laguerre filter 

The Laguerre filters are first-order lag filters with one real pole. They are, therefore, more 
appropriate for well damped processes. The Laguerre filters are given by 

where p is the estimated pole which is related to the time constant, r, and the sampling 
interval T s of the system by 

P = e- {TjT) (13) 

Kautz filter 

Kautz filters allow the incorporation of a pair of conjugate complex poles. They are, 
therefore, effective for modeling weakly damped processes. The Kautz filters are defined by 



q + a(b-l)q-b 



h i -- f^ ){q - a \ ^ M ,i) (15) 

q +a(b~l)q-b 



where 
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1 -bq 2 + a(b -\)q + 1 Y 
q 2 + a(b-l)-b 



(16) 



g(a,b,q,i) = 

-1 < a < 1 and -Kb<l n = l,2, ... 

Generalized orthonormal basis filter 

Van den Hof, et ah, (1995) introduced the generalized orthonormal basis filters and showed 
the existence of orthogonal functions that, in a natural way, are generated by stable linear 
dynamic systems and that form an orthonormal basis for the linear signal space 1" . Ninness 
& Gustafsson (1997) unified the construction of orthonormal basis filters. The GOBF filters 
are formulated as 

Jl-|», I 2 44(1 -p*q) 

iq-Pi) j=\ ii-Pj) 

where p = \pj : j = 1, 2, 3, . . . } is an arbitrary sequence of poles inside the unit circle appearing 
in complex conjugate pairs. 

Markov-OBF 

When a system involves a time delay and an estimate of the time delay is available, Markov- 
OBF can be used. The time delay in Markov-OBF is included by placing some of the poles at 
the origin (Heuberger, et al., 1995). For a SISO system with time delay equal to d samples, 
the basis function can be selected as: 

fi=z i fori =1,2, ...,d (18) 

fi + Al>P) = -, -\{- — ~ z fori = l,2,...,N (19) 

Patwardhan and Shah (2005) presented a two-step method for estimating time delays from 
step response of GOBF models. In the first step, the time delays in all input-output channels 
are assumed zero and the model is identified with GOBF. In GOBF models, the time delay is 
approximated by a non-minimum phase zero and the corresponding step response is an 
inverse response. The time delay is then estimated from a tangent drawn at the point of 
inflection. 

2.2 Estimation of GOBF poles 

Finding an appropriate estimate of the poles for the filters is an important step in estimating 
the parameters of the OBF models. Arbitrary choice of poles may lead to a non- 
parsimonious model. Van den Hof, et al., (2000) showed that for a SISO system with poles 
{aj : | aj | < 1 for j =1, 2 , ..., n}, the rate of convergence of the model parameters is 
determined by the magnitude of the slowest Eigen value. 
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p = maxYJ 



a j-Pk 



1 - Pk a j 



(20) 



where pt= arbitrary poles. 



Therefore, a good approximation by a small number of parameters can be obtained by 
choosing a basis for which p is small. It is shown that the poles determined by Van den Hof et 
al. method closely match the dominant poles of the system (Heuberger, et al., 2005; Wahlberg, 
1991). Lemma, et al., (2011) proposed a systematic way to estimate the dominant poles and 
time delays of a system from the input-output identification test data. An OBF model is first 
developed with randomly chosen real poles and generalized orthonormal basis filters with 10- 
12 terms. The model is simulated to get a noise free step response of the system. One or two 
poles of the system are estimated from the resulting noise free step response of the OBF model 
and it is also observed whether the system is weakly damped or not. This process can be 
repeated until some convergence criterion is fulfilled. The procedure normally converges after 
two or three iterations. The procedure is iiterations and is illustrated in Example 1. 

2.3 Model parameter estimation 

In OBF models, the output can be expressed as a linear combination of the input sequence 
filtered by the respective filters. For a finite number of parameters, from (9) we get 



y(k) = l t fM u { k ) + kfiilMk) + ■■■ + Un(lM k ) 



(21) 



Equation (21) is not linear in its parameters and therefore estimation of parameters using 
linear least square method is impossible. However, it can be modified such that it is linear in 
parameters, as 



y(k) = l 1 u fl (k) + l 2 u f2 (k) + ... + l nUfn (k) 



where u fi{k) is the filtered input given by 



u fi( k ) = f,( c lH k ) 



(22) 



(23) 



Once the dominant poles of the system and the types of filters are chosen, the filters 
/1//2, • • -,/n are fixed. The filtered inputs, Uf,, are determined by filtering the input sequence with 
the corresponding filter. For an OBF model with n parameters, the prediction can be started 
from the n th instant in time. Equation (22) can be expanded and written in matrix form as 



2/„+l 

Vn+2 



Vn 



u fl {n) u f2 (n-l) 

Ur^n + 1) Ur 2 { n ) 



u fl (N-l) u f2 (N-2) 



«/h(2) 



UfriN-n) 



(24) 
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where N is the future time instant. 

Equation (24) in vector-matrix notation is given by 

y = X0 (25) 

where# = YL, l 2 , ■ ■■,'„] is the parameter vector, y is the output vector y =[y n +i, yn+2/--/ vn] 
and X is the regressor matrix given by 

u fl (n) u f2 (n-l) ... UfJJ) 

u fl (n + l) u f2 (n) ... iifnil) 



X: 



u fl (N-l) u f2 (N-2) 



u,in(N-n) 



(26) 



Since (25) is linear in parameters, the model parameters can be estimated using linear least 
square formula (27). 



= (X T Xy 1 X T y 



(27) 



Algorithm 1 



1 . Use GOBF structure and two randomly selected stable poles and develop (6 to 12) 
sequence of GOBF filters 

2. Develop the regressor matrix (26) using the filters developed at step (1) and the input 
sequence u(k) 

3. Use the linear least square formula (27) to estimate the model parameters 

4. Make a better estimate of the poles of the system from the step response of the GOBF 
model 

5. Repeat steps 1 to 4 with the new pole until a convergence criterion is satisfied 

The Percentage Prediction Error (PPE) can be a good convergence criterion. 

Example 1 

An open loop identification test for SISO system is carried out and the input-output data 
shown in Figure 1 is obtained. A total of 4000 data points are collected at one minute 
sampling interval with the intention of using 3000 of them for modelling and 1000 for 
validation. Develop a parsimonious OBF model using the data. No information is available 
about the pole of the system. 

Since there is no information about the poles of the system, two poles: 0.3679 and 0.9672 are 
arbitrarily chosen for the first iteration of the model. A GOBF model with six terms (you can 
choose other numbers and compare the accuracy if you need) is first developed with these 
two poles alternating. Note that, once the poles, type of filter, i.e., GOBF and the number of 
terms is fixed the filters are fixed and the only remaining value to determine the model is 
the model parameters. 
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Fig. 1. Input-output data used for identification 

To estimate the model parameters the regressor matrix is developed and used together with 
the plant measured output y(k) in the least square formula (27) to find the model 
parameters: 

[-0.2327 0.8733 -0.2521 0.8854 -0.8767 -0.2357] 

The percentage prediction error (PPE) is found to be 9.7357. For the second iteration, the 
poles of the system are estimated from the noise free step-response of the GOBF model 
shown in Figure 2. 




Fig. 2. Step response of the noise free OBF model developed in the first iteration 
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The OBF model parameters, the PPE and poles estimated at each iteration are presented in 
Table 1. 



Iterations PPEs 



Poles 



Model Parameters 



9.7357 [0.3679 0.9672] [0.2327 0.8733 -0.2521 0.8854 -0.8767 -0.2357] 
9.5166 [ 0.9467 0.9467] [0.7268 0.7718 0.4069 -0.5214 0.0273 0.0274] 
9.5149 [0.9499 0.9306] [0.4992 0.97810.4723-0.3377 0.1387-0.0305] 



Table 1. The results of the OBF iterative identification method 

Note that the parameters in the last iteration together with the OBF filters determine the 
model of the plant. The model accuracy is judged by cross validation. Figure 3 shows the 
measured output data for sampling instants 3001 to 4000 (this data is not used for 
modelling) and the result of the OBF simulation for the plant input for the instants 3001 to 
4000. 

3. BJ- Type models by combining OBF with conventional noise model 
structures 

In Example 1, we developed a GOBF model to capture the deterministic component of the 
plant. The residual of the model however was just discarded. In reality, this residual may 
contain useful information about the plant. However, as it is already noted, conventional 
OBF models do not include noise models. Patwardhan and Shah ( 2005) showed that the 
regulatory performance of MPC system improves significantly by including a noise model 
to the OBF simulation model. In their work, the residual of the OBF model is whitened with 



measured data 
■ OBF model simulation 



2 - 

1 - 

* °i 
-1 




-2 -''! tf> V n.VlA 
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k 

Fig. 3. Validation of the final GOBF model with 6 parameters. 

Auto Regressive (AR) noise model. The AR noise model is parameterized in terms of OBF 
parameters and a minimal order state space model was realized. In this section, an 
integrated approach for developing BJ models with an OBF plant model and AR or ARMA 
noise model is presented. 
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3.1 Model structures 

The BJ model structure is known to be the most flexible and comprehensive structure of the 
conventional linear models(Box & Jenkins, 1970). 

y {k) = ^lu(k) + ^e(k) (28) 

In (28) B(q)/F(q) describes the plant model whereas C(q)/D(q) describes the noise model. 
The BJ-type model structure proposed by Lemma, et ah, (2010) is obtained by replacing the 
plant model structure with OBF model structure. First, the OBF-AR structure, i.e., with 
C(q)=l is discussed then the OBF-ARMA structure is discussed. 

The OBF-AR model structure assumes an OBF and AR structures for the plant and noise 
transfer functions, respectively. 

y(k) = G 0BF (l)u(k) + -^-e(k) (29) 

D(q) 

The OBF-ARMA structure has more flexible noise model than the OBF-AR structure as 
given by (30). 

y(k) = G OBF (q)u(k) + ^ie(k) (30) 

D(q) 

3.2 Estimation of model parameters 

The model parameters of both OBF-AR and OBF-ARMA structures are estimated based on 
the prediction error method as explained below. 

Estimation of parameters of OBF-AR model 

The prediction error e(k) is defined as 

e(k) = y(k)-y(k\k-l) (31) 

Introducing the prediction error (31) in (29) and rearranging leads to 

y(k | k - 1) = D(q)C 0BF (q)u(k) + (1 - D(q))y(k) 

Assuming that the noise sequence is uncorrelated to the input sequence, the parameters of 
the OBF model can be estimated separately. These parameters can then be used to calculate 
the OBF simulation model output using (32). 

y bf( k ) = G oBF(i) u ( k ) ( 33 ) 

Inserting (33) in (32) 

y(k | k - 1) = D(q)y ohf (k) + (1 - D(q))y(k) (34) 
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Equation (34) is linear in parameters since y bf(k) is already known. With D(q) monk, (34) 
can be expanded and rearranged to yield 



y(k\k-l) = y obf (k)-d 1 r(k-l)-d 2 r(k-2)-...-d n r(k-n) 



(35) 



where 



n is the order of the polynomial D(q) 
r(i) = y(i)-y bf(i) 

Note that r(i) represents the residual sequence of the output sequence y(k) of the system 
from the OBF model output y bj(k). The model parameters in (35) can be calculated by the 
linear least square formula (27) with the regressor matrix given by (36). 



y bf( n ) -r(n-l) -r(n-2)- ... -r(l) 
y bf( n + 1 ) - r ( n ) -r(n-l)- ...-r(2) 



X: 



y obf (N) -r(N-l) -r(N -2) -. . . -r(N -n) 



(36) 



where n = no. 



The step-by-step procedure for estimating the OBF-AR model parameters, explained above, 
is outlined in Algorithm 2. 

Algorithm 2 

1. Develop a parsimonious OBF model 

2. Determine the output sequence of the OBF model y u (k) for the corresponding input 
sequence u(k) 

3. Determine the residuals of the OBF model r(k ) = y(k) - y bf(k) 

4. Develop the regression matrix X given by (36) 

5. Determine the parameters of the noise model using (27) enforcing monic condition, i.e., 
do = 1. 

Estimation of parameters of OBF-ARMA model 

The OBF-ARMA structure is given by (28) 



y(*) = G 0BF fa)«(k) + §4e(*) 
D(q) 

Substituting the prediction error (31) in (28) and rearranging yields 

C(q)y(k | k -1) = D(q)G OBF (q)u(k) - D(q)y(k) + C(q)y(k) 



(28) 



(37) 
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As in the case of OBF-AR model, if the noise sequence is uncorrelated with the input 
sequence, the OBF model parameters can be calculated separately and be used to calculate 
the simulation model output y tt(k) using (33). 

Introducing (33) in (37) results in 

C(q)y(k | k - 1) = D(q)y obf (k) - D(q)y(k) + C(q)y(k) (38) 

Expanding and rearranging (37) we get 

y(k\k-l) = y obf (k)-d 1 r(k-l)-d 2 r(k-2)-...-d m r(k-m) + 
c x e(k - 1) + c 2 e(k - 2) + . . . + c n e(k - n) 

The parameter vector and the regressor matrix are derived from (39) and are given by (40) 
and (41) 

6 = {d 1 d 2 ...d m c 1 c 2 ...c n \ T (40) 

where n = nc, the order of the polynomial C(q) 
m = no, the order of the polynomial D(q) 
mx=max (m, n)+l 



X = 



y oh Amx) -r(mx-l) -r(mx-2) -...-r(mx-n) e(mx-l) e(mx-2) .. .e(mx-m) 
y oh f(mx + l) -r(mx) -r(mx-l) -...-r(mi-u + l) e(mx) e(mx-l). . .e(mx-m + l) 



(41) 



y ohf (N) -r(N-l) -r{N-2)-... -r(N-n + l) e(N-l) e{N-2)... e{N-m + T) 

y=[y(mx) y(mx + l) ... y(N)] T (42) 

Equation (39) in the form shown above appears a linear regression. However, since the 
prediction error sequence, e(k-i), itself is a function of the model parameters, it is nonlinear 
in parameters. To emphasize the significance of these two facts such structures are 
commonly known as pseudo-linear(Ljung, 1999; Nelles, 2001). The model parameters can be 
estimated by either a nonlinear optimization method or an extended least square method 
(Nelles, 2001). The extended least square method is an iterative method where the 
prediction error sequence is estimated and updated at each iteration using the prediction 
error of OBF-ARMA model. A good initial estimate of the prediction error sequence is 
obtained from the OBF-AR model. The parameters for the noise model are estimated using 
the linear least square method with (40) and (41) as parameters vector and regressor matrix, 
respectively. From the derivation, it should be remembered that all the poles and zeros of 
the noise models should be inside the unit circle and both the numerator and denominator 
polynomials should be monic. If an OBF-AR model with a high-order noise model can be 
developed, the residuals of the OBF-AR model will generally be close to white noise. In such 
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cases, the noise model parameters of the OBF-ARMA model can be estimated using linear 
least square method in one step. The step-by-step procedure for estimating OBF-ARMA 
model parameters is outlined in Algorithm 3. 

Algorithm 3 

1. Develop a parsimonious OBF model 

2. Determine the OBF simulation model output y bf(k) for the corresponding input 
sequence u(k) 

3. Determine the residual of the simulation model r(k)= y(k)- y bf (k) 

4. Develop OBF-AR prediction model 

5. Determine the residual of the OBF-AR model, e(k) 

6. Use j/obf (k), r(k) and e(k) x e(k) to develop the regressor matrix (40) 

7. Use the linear least square formula (27) to estimate the parameters of the OBF ARMA 
model 

8. Re-estimate the prediction error e(k) = y(k) - y(k) irom the residual of OBF-ARMA 
model developed in step 7 

9. Repeat steps 6 to 8 until convergence is achieved 

Convergence criteria 

The percentage prediction error (PPE) can be used as convergence criteria, i.e., stop the 
iteration when the percentage prediction error improvement is small enough. 

i(y(k)-mf 

PPE = -*=* x 100 

where y represents the mean value of measurements { y(k) } and y(k) predicted value of 

y(*)- 

3.3 Multi-step ahead prediction 

Multi-step ahead predictions are required in several applications such as model predictive 
control. In this section multi-step ahead prediction equation and related procedures for both 
OBF-AR and OBF-ARMA are derived. 

Multi-step ahead prediction using OBF-AR model 

Using (33) in (29) the OBF-AR equation becomes 

y {k ) = y ( k) + J-e(k) (43) 

D(q) 

f-step ahead prediction is obtained by replacing k with k + i 

y(k + i) = y„ bf (k + i) + -—e(k + i) (44) 

D(q) 
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To calculate the f-step ahead prediction, the error term should be divided into current and 
future parts as shown in (45). 

y(k + i) = y obf (k + i) + ^-e(k) + E i (q)e(k + i) (45) 

D{q) 

The last term in (45) contains only the future error sequence which is not known. However, 
since e(k) is assumed to be a white noise with mean zero, (45) can be simplified to 

y{k + i\k) = y obf {k + i) + ^\e{k) (46) 

D{q) 

F{ and E; are determined by solving the Diophantine equation (47) which is obtained by 
comparing (44) and (45) 

1 :Ei{q)+ ±!m (47) 



D(q) D(q) 

Equation (46) could be taken as the final form of the i-step ahead prediction equation. 
However, in application, since e(k) is not measured the equation cannot be directly used. 
The next steps are added to solve this problem. 

Rearranging (43) to get 

J_ e (fc) = y(fc)-y (fc) (48) 

D(q) 

Using (48) in (46) to eliminate e(k) 

y(k + i\k) = y obf (k + i) + FM){y{k)-y ob m) (49) 

Rearranging (49) 

y(k + i\k) = y obf (k + i)(l-F i (q)q- i ) + F i (q)y(k) (50) 

Rearranging the Diophantine equation (47) 

(l-^(?)) = D(<?)£ ( (?) (51) 

Using (51) in (50) 

y(k + i\k) = E i (q)D(q)y obf (k + i)+F i (q)y(k) (52) 

Equation (52) is the usable form of the multi-step ahead prediction equation for the OBF-AR 
model. Given an OBF-AR model, the solution of the Diophantine equation to get Ej and F\ 
and the prediction equation (52) forms the procedure for i-step ahead prediction of the OBF- 
AR model. 
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Multi-step ahead prediction using OBF-ARMA model 

Using (33) in (30) the OBF-ARMA equation becomes 

y(k) = y (k) + ^\e(k) (53) 

D(q) 

f-step ahead prediction is obtained by replacing k with k + i 

y(k + i) = y obf (k + i) + ^- e (k + i) (54) 

D(q) 

To calculate the f-step ahead prediction, the error term should be divided into current and 
future parts. 

y(k + = y obf (k + + ^le(k) + EMW + ( 55 ) 

Since e(k) is assumed to be a white noise with mean zero, the mean of Ei(q) e(k+i) is equal to 
zero, and therefore (55) can be simplified to 

y(k + i\k) = y obf (k + i) + ^e(k) (56) 

D(q) 

F{ and E, are determined by solving the Diophantine equation (57) which is obtained by 
comparing (54) and (56) 

m = EM + ±ilM> (57) 

D(q) D(q) 

Rearranging (57) 

Using (58) in (56) to eliminate e(k) 

y(k + i\k) = y obf (k + i) + 5M(y(k)-y obf (k)) (59) 



Rearranging (59) 



y{ k + i \ k ) = y b f ( k + i ) 

Rearranging the Diophantine equation (60) 



i-^f }+s®m (60) 



C(q) I C{q) 



(61) 
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Using (61) in (60) results in the final usable form of the i-step ahead prediction for OBF- 
ARMA model. 



y(k + i\k) 



C(q) C(q) 



(62) 



Since y bf (k+i) is the output sequence of the simulation OBF model, if the OBF model 
parameters are determined its value depends only on the input sequence u(k+i). Therefore, 
the f-step ahead prediction according to (62) depends on the input sequence up to instant k+i 
and the output sequence up to instant k. 

Multiple-Input Multiple-Output (MIMO) systems 

The procedures for estimating the model parameters and i-step ahead prediction can be 
easily extended to MIMO systems by using multiple-MISO models. First, a MISO OBF 
model is developed for each output using the input sequences and the corresponding 
orthonormal basis filters. Then, AR model is developed using y y(k) and the residual of the 
OBF simulation model. The OBF-ARMA model is developed in a similar manner, with an 
OBF model relating each output with all the relevant inputs and one ARMA noise model for 
each output using Algorithm (Lemma, et al., 2010). 

Example 2 

In this simulation case study, OBF-AR and OBF-ARMA models are developed for a well 
damped system that has a Box-Jenkins structure. They are developed with various orders 
and compared within themselves and with each other. The system is represented by (63). 
Note that both the numerator and denominator polynomials of the noise model are monic 
and their roots are located inside the unit circle. 



-i 



-2 



y{k)=q 



1-1.3^+0.42^ l + 0.6 ? 

l-235q~ x +2165q~ 1 -Qfillq* \-1.15q- 1 



0.58(7 



^e(k) 



(63) 



An identification test is simulated on the system using MATLAB and the input-output 
sequences shown in Figure 4 is obtained. 




0.1 



a 



-0.1 



500 1000 1500 2000 2500 3000 

k 
Fig. 4. Input-output data sequence generated by simulation of (63) 
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The mean and standard deviations of the white noise, e(k), added to the system are 0.0123 
and 0.4971, respectively, and the signal to noise ratio (SNR) is 6.6323 . The input signal is a 
pseudo random binary signal (PRBS) of 4000 data points generated using the 'idinput' 
function in MATLAB with band [0 0.03] and levels [-0.1 0.1]. Three thousand of the data 
points are used for model development and the remaining 1000 for validation. The 
corresponding output sequence of the system is generated using SIMULINK with a 
sampling interval of 1 time unit. 

OBF-AR model 

First a GOBF model with 6 parameters and poles 0.9114 and 0.8465 is developed and the 
model parameters are estimated to be [3.7273 5.6910 1.0981 -0.9955 0.3692 -0.2252] using 
Algorithm 1. The AR noise model developed with seven parameters is given by: 

(64) 



D(q) 1 - 1.7646( ? _1 + 1.6685(7 " 2 " 1-0119(7 " 3 + 



4 - 0.3 154^ - 5 + 0.1435(7 ■" - 0.0356(7 " 7 



The spectrum of the noise model of the system compared to the spectrum of the model for 3, 
5 and 7 parameters is shown in Figure 5. The percentage predication errors of the spectrums 
of the three noise models compared to spectrum of the noise model in the system is given in 
Table 2. 



n D 


PPE 


3 

5 
7 


54.3378 
1.5137 
0.9104 



Table 2. PPE of the three AR noise models of system 




Fig. 5. Spectrums of the AR noise models for no = 2, 5 and 7 compared to the noise transfer 
function of system 
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It is obvious from both Figure 4 and Table 2 that the noise model with no = 7 is the closest to 
the noise transfer function of the system. Therefore, this noise model together with the 
GOBF model described earlier form the OBF-AR model that represent the system. 

4. Closed loop identification using OBF-ARX and OBF-ARMAX structures 

When a system identification test is carried out in open loop, in general, the input 
sequence is not correlated to the noise sequence and OBF model identification is 
carried out in a straight forward manner. However, when the system identification test is 
carried out in closed loop the input sequence is correlated to the noise sequence and 
conventional OBF model development procedures fail to provide consistent model 
parameters. 

The motivation for the structures proposed in this section is the problem of closed-loop 
identification of open-loop unstable processes. Closed-loop identification of open-loop 
unstable processes requires that any unstable poles of the plant model should be shared by 
the noise model H(q) otherwise the predictor will not be stable. It is indicated by both Ljung 
(1999) and Nelles (2001) that if this requirement is satisfied closed-loop identification of 
open-loop unstable processes can be handled without problem. In this section, two different 
linear structures that satisfy these requirements and which are based on OBF structure are 
proposed. While the proposed models are, specially, effective for developing prediction 
model for open-loop unstable process that are stabilized by feedback controller, they can be 
used for open-loop stable process also. These two linear model structures are OBF-ARX and 
OBF- ARM AX structures. 

4.1 Closed-loop identification using OBF-ARX model 

Consider an OBF model with ARX structure given by (65) 

m-^um^ (65) 

Rearranging (65) 

y(k\k-l) = G 0BF (q)-(l-A(q))y(k) (66) 

With A(q) monic (66) can be expanded to 

y(k | k - 1) = G OBF (q) - a iy {k - 1) - a 2 y(k - 2) - a m y(k - m) (67) 

Note that, (67) can be further expanded to 

y(k\k-l) = l 1 u fl (k) + l 2 u f2 (k) + ... + l m u fn ,(k)- 

(681 
aiy(k -1) - a 2 y(k -2) - ...- a„y(k -n) 

Therefore, the regressor matrix for the OBF-ARX structure is given by 
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X- 



Uf^mx) Uf 2 (inx-])---Ut tn (mx-m) —y(mx — T)—y(nx—2)... — y(mx — ri) 



(69) 



u fl (N) u f2 {N-l)... Ufin {N-m) -y(N-l) - y (N-2)...-y(N-n) 

where m = order of the OBF model 

n = order of A(q) 
mx = max (n, m) + 1 
Ufi= input u filtered by the corresponding OBF filter^ 

The parameters are estimated using (69) in the least square equation (27). Note that in using 
(27) the size of y must be from mx to N. 

4.2 Multi-step ahead prediction using OBF-ARX model 

Consider the OBF-ARX model 



... Vobf( k ) 1 ... 
A(q) A(q) 

f-step ahead prediction is obtained by replacing k with k + i 

yobf( k + i ) i 



y(k + i)-- 



A{q) A(q) 



e(k + i) 



(70) 



(71) 



To calculate the f-step ahead prediction, the noise term can be divided into current and 
future parts. 



., ., y bf{ k + i) F,(q) 
A{q) A{q) 



(72) 



Since e(k) is assumed to be a white noise with mean zero, the mean of Ei(q) e(k+i) is equal to 
zero (72) can be simplified to 



... .... Vobfik + i) F,(q) 

y(k + 1 1 k) = — + - L1LL e(k) 



A(q) A(q) 



(73) 



On the other hand rearranging (71) 



y(k + i)- 



y„bf( k +i) 



e(k + i) 



<rm) 



£,(<?) 



A(q) \ A(q) 

Comparing (70) and (73), F, and E, can be calculated by solving the Diophantine equation 



(74) 
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1 -M)*^ (75) 



Mi) Mi) 

Rearranging (70) 



1 „, „. Vobf( k ) 

e(k) = y{k) — -j— (76) 



Ml) Ml) 

Using (76) in (73) to eliminate e(k) 

... . IM Vobf(k + i) „,/,,, 2A*/( fc ) 

y( fc+ '|fe)= T7 : +5(q) y(*)- 



A(q) "'V Afo) 

^ + 4ir^) +w) (77) 

Rearranging the Diophantine equation (76) 

^—-^S? (78) 

Finally using (78) in (77), the usable form of the f-step ahead prediction formula, (79), is 
obtained. 

y(k + i\k) = E i (q)y ohf (k + i) + F i (q)y(k) (79) 

Note that in (79), there is no any denominator polynomial and hence no unstable pole. 
Therefore, the predictor is stable regardless of the presence of unstable poles in the OBF- 
ARX model. It should also be noted that, since y bf (k+i) is the output sequence of the 
simulation OBF model, once the OBF model parameters are determined its value 
depends only on the input sequence u(k+i). Therefore, the f-step ahead prediction according 
to (79) depends on the input sequence up to instant k+i and the output sequence up to 
instant k. 

4.3 Closed-loop identification using OBF-ARMAX model 

Consider the OBF model with ARMAX structure 

{k) = G^M u{k)+ m e{k) (80) 

Mi) Mi) 

Rearranging (80) 

y(k | k - 1) = G OBF (q) - (1 - A{q))y{k) + (C(q) - l)e(k) (81) 

With A(q) and C(q) monic, expanding (74) 
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y(k | k-1) = hu fl {k) + l 2 u f2 (k) + ... + InUfJJc) + 

-a l y(k-l)-a 2 y(k-2)-.,.-a„y(k-n) + 
c t e(k - 1) + c 2 e(k -2) + .,. + c n e(k-n) 

From (83) the regressor matrix is formulated for orders m, n, p 

Ufi(mx) Uf2(rnx — T)...Uf m (mx — in) -y(mx — T)—y(nx—2)...-y(mx-ri) 

X= ... 

u fl (N) u f2 {N-l)... Ufm {N-m) -y(N-l) -y(N-2)...-y(N-n) 

-e(mx-l) -e(mx-2)...-e(mx-p) 



(82) 



(83) 



-e(N-l) -e(N-2)...-e(N-p) 

where m = order of the OBF model 

n = order of the A(q) 

p = order of C(q) 

mx = max (n,m,p) + l 

Ufi= input u filtered by the corresponding OBF filter fi 

e(i) = the prediction error 

To develop an OBF-ARMAX model, first an OBF-ARX model with high A(q) order is 
developed. The prediction error is estimated from this OBF-ARX model and used to form 
the regressor matrix (83). The parameters of the OBF-ARMAX model are, then, estimated 
using (83) in (27). The prediction error, and consequently the OBF-ARMAX parameters can 
be improved by estimating the parameters of the OBF-ARMAX model iteratively. 

Multi-step ahead prediction using OBF-ARMAX model 

A similar analysis to the OBF-ARX case leads to a multi-step ahead prediction relation given 
by 



y(k + r\k) = ^f±y ohf (k + r) + ^±y(k) 
C{q) C(q) 

where F, and E, are calculated by solving the Diophantine equation 



A(q) 



A{q) 



(84) 



(85) 
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When OBF-ARMAX model is used for modeling open-loop unstable processes that are 
stabilized by a feedback controller, the common denominator A(q) that contains the unstable 
pole does not appear in the predictor equation, (84). Therefore, the predictor is stable 
regardless of the presence of unstable poles in the OBF-ARMAX model, as long as the noise 
model is invertible. Invertiblity is required because C(q) appears in the denominator. It 
should also be noted that, since y bf (k+i) is the output sequence of the OBF simulation 
model, once the OBF model parameters are determined its value depends only on the input 
sequence u(k+i). Therefore, the i-step ahead prediction according to (84) depends on the 
input sequence up to instant k+i and the output sequence only up to instant k. 

5. Conclusion 

OBF models have several characteristics that make them very promising for control relevant 
system identification compared to most classical linear models. They are parsimonious 
compared to most conventional linear structures. Their parameters can be easily calculated 
using linear least square method. They are consistent in their parameters for most practical 
open-loop identification problems. They can be used both for open-loop and closed-loop 
identifications. They are effective for modeling system with uncertain time delays. While the 
theory of linear OBF models seems getting matured, the current research direction is in OBF 
based non-linear system identification and their application in predictive control scenario. 
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